[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 405277950 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] MTRR default type: write-back [ 0.000000] MTRR fixed ranges enabled: [ 0.000000] 00000-9FFFF write-back [ 0.000000] A0000-BFFFF uncachable [ 0.000000] C0000-FFFFF write-protect [ 0.000000] MTRR variable ranges enabled: [ 0.000000] 0 base 0000C0000000 mask 3FFFC0000000 uncachable [ 0.000000] 1 disabled [ 0.000000] 2 disabled [ 0.000000] 3 disabled [ 0.000000] 4 disabled [ 0.000000] 5 disabled [ 0.000000] 6 disabled [ 0.000000] 7 disabled [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] BRK [0x139601000, 0x139601fff] PGTABLE [ 0.000000] BRK [0x139602000, 0x139602fff] PGTABLE [ 0.000000] BRK [0x139603000, 0x139603fff] PGTABLE [ 0.000000] BRK [0x139604000, 0x139604fff] PGTABLE [ 0.000000] BRK [0x139605000, 0x139605fff] PGTABLE [ 0.000000] BRK [0x139606000, 0x139606fff] PGTABLE [ 0.000000] BRK [0x139607000, 0x139607fff] PGTABLE [ 0.000000] BRK [0x139608000, 0x139608fff] PGTABLE [ 0.000000] BRK [0x139609000, 0x139609fff] PGTABLE [ 0.000000] BRK [0x13960a000, 0x13960afff] PGTABLE [ 0.000000] BRK [0x13960b000, 0x13960bfff] PGTABLE [ 0.000000] BRK [0x13960c000, 0x13960cfff] PGTABLE [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] On node 0 totalpages: 1076588 [ 0.000000] DMA zone: 64 pages used for memmap [ 0.000000] DMA zone: 158 pages reserved [ 0.000000] DMA zone: 3998 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 12224 pages used for memmap [ 0.000000] DMA32 zone: 782286 pages, LIFO batch:63 [ 0.000000] Normal zone: 4536 pages used for memmap [ 0.000000] Normal zone: 290304 pages, LIFO batch:63 [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] ACPI: IRQ0 used by override. [ 0.000000] ACPI: IRQ5 used by override. [ 0.000000] ACPI: IRQ9 used by override. [ 0.000000] ACPI: IRQ10 used by override. [ 0.000000] ACPI: IRQ11 used by override. [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] pcpu-alloc: s221184 r8192 d28672 u524288 alloc=1*2097152 [ 0.000000] pcpu-alloc: [0] 0 1 2 3 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001000] hpet clockevent registered [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002331] x2apic enabled [ 0.003004] Switched APIC routing to physical x2apic. [ 0.004011] kvm-guest: setup PV IPIs [ 0.006869] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008012] pid_max: default: 32768 minimum: 301 [ 0.009134] LSM: Security Framework initializing [ 0.010053] Yama: becoming mindful. [ 0.011037] SELinux: Initializing. [ 0.012061] *** VALIDATE selinux *** [ 0.020655] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024568] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025125] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026116] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028085] *** VALIDATE tmpfs *** [ 0.029457] *** VALIDATE proc *** [ 0.030150] *** VALIDATE cgroup *** [ 0.031005] *** VALIDATE cgroup2 *** [ 0.033139] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034136] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036025] Spectre V2 : User space: Vulnerable [ 0.037006] Speculative Store Bypass: Vulnerable [ 0.039897] debug: unmapping init [mem 0xffffffffa7859000-0xffffffffa7860fff] [ 0.041819] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.042544] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.043017] ... version: 2 [ 0.044006] ... bit width: 48 [ 0.045008] ... generic registers: 4 [ 0.045988] ... value mask: 0000ffffffffffff [ 0.046013] ... max period: 00007fffffffffff [ 0.047010] ... fixed-purpose events: 3 [ 0.048007] ... event mask: 000000070000000f [ 0.049298] rcu: Hierarchical SRCU implementation. [ 0.051238] smp: Bringing up secondary CPUs ... [ 0.052441] x86: Booting SMP configuration: [ 0.053019] .... node #0, CPUs: #1 #2 #3 [ 0.056127] smp: Brought up 1 node, 4 CPUs [ 0.058011] smpboot: Max logical packages: 1 [ 0.059026] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.189532] node 0 deferred pages initialised in 130ms [ 0.192621] devtmpfs: initialized [ 0.193166] x86/mm: Memory block size: 128MB [ 0.195715] gcov: version magic: 0x41383552 [ 0.198170] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.200091] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.202372] pinctrl core: initialized pinctrl subsystem [ 0.204804] ************************************************************* [ 0.206013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.208012] ** ** [ 0.210011] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.211008] ** ** [ 0.213013] ** This means that this kernel is built to expose internal ** [ 0.215012] ** IOMMU data structures, which may compromise security on ** [ 0.217011] ** your system. ** [ 0.219012] ** ** [ 0.221013] ** If you see this message and you are not debugging the ** [ 0.223015] ** kernel, report this immediately to your vendor! ** [ 0.225011] ** ** [ 0.227010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.229018] ************************************************************* [ 0.231702] NET: Registered protocol family 16 [ 0.233543] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.236048] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.238078] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.241415] cpuidle: using governor menu [ 0.243257] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.245440] PCI: Using configuration type 1 for base access [ 0.247107] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.254098] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.256022] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.259116] cryptd: max_cpu_qlen set to 1000 [ 0.261236] ACPI: Added _OSI(Module Device) [ 0.263011] ACPI: Added _OSI(Processor Device) [ 0.264021] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.265025] ACPI: Added _OSI(Processor Aggregator Device) [ 0.269463] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.275383] ACPI: Interpreter enabled [ 0.276038] ACPI: PM: (supports S0 S3 S4 S5) [ 0.277010] ACPI: Using IOAPIC for interrupt routing [ 0.279091] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.282412] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.290391] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.292021] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.294015] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.297071] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.301043] acpiphp: Slot [2] registered [ 0.302099] acpiphp: Slot [5] registered [ 0.303096] acpiphp: Slot [6] registered [ 0.304144] acpiphp: Slot [7] registered [ 0.306085] acpiphp: Slot [8] registered [ 0.307117] acpiphp: Slot [9] registered [ 0.309069] acpiphp: Slot [10] registered [ 0.310109] acpiphp: Slot [3] registered [ 0.311047] acpiphp: Slot [4] registered [ 0.312067] acpiphp: Slot [11] registered [ 0.313053] acpiphp: Slot [12] registered [ 0.315070] acpiphp: Slot [13] registered [ 0.316105] acpiphp: Slot [14] registered [ 0.318139] acpiphp: Slot [15] registered [ 0.319105] acpiphp: Slot [16] registered [ 0.321097] acpiphp: Slot [17] registered [ 0.322093] acpiphp: Slot [18] registered [ 0.323082] acpiphp: Slot [19] registered [ 0.325100] acpiphp: Slot [20] registered [ 0.326087] acpiphp: Slot [21] registered [ 0.327072] acpiphp: Slot [22] registered [ 0.328063] acpiphp: Slot [23] registered [ 0.329071] acpiphp: Slot [24] registered [ 0.331064] acpiphp: Slot [25] registered [ 0.332084] acpiphp: Slot [26] registered [ 0.334113] acpiphp: Slot [27] registered [ 0.335095] acpiphp: Slot [28] registered [ 0.336062] acpiphp: Slot [29] registered [ 0.338070] acpiphp: Slot [30] registered [ 0.339081] acpiphp: Slot [31] registered [ 0.340049] PCI host bridge to bus 0000:00 [ 0.342018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.344052] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.346018] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.349020] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.351012] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.353017] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.355158] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.357936] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.361071] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.369012] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.373759] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.377012] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.379012] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.381013] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.383520] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.386578] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.388036] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.390777] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.395013] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.407014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.411904] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.417012] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.422013] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.427022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.440021] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.453482] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.460014] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.466016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.480017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.495082] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.502018] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.508014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.529018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.541118] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.547014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.553023] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.569019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.580130] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.585015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.591011] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.604014] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.613788] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.617012] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.621987] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.635019] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.645712] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.648421] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.650228] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.652306] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.655341] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.661123] iommu: Default domain type: Passthrough [ 0.662480] SCSI subsystem initialized [ 0.664169] ACPI: bus type USB registered [ 0.665098] usbcore: registered new interface driver usbfs [ 0.667075] usbcore: registered new interface driver hub [ 0.669081] usbcore: registered new device driver usb [ 0.670107] pps_core: LinuxPPS API ver. 1 registered [ 0.672007] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.675047] PTP clock support registered [ 0.676125] EDAC MC: Ver: 3.0.0 [ 0.677320] PCI: Using ACPI for IRQ routing [ 0.679019] PCI: pci_cache_line_size set to 64 bytes [ 0.679364] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] [ 0.679378] e820: reserve RAM buffer [mem 0xbffce000-0xbfffffff] [ 0.679382] e820: reserve RAM buffer [mem 0x146e00000-0x147ffffff] [ 0.679869] NetLabel: Initializing [ 0.681011] NetLabel: domain hash size = 128 [ 0.682009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.684081] NetLabel: unlabeled traffic allowed by default [ 0.686135] vgaarb: loaded [ 0.687288] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.690013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.694008] clocksource: Switched to clocksource kvm-clock [ 0.787706] VFS: Disk quotas dquot_6.6.0 [ 0.789279] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.791192] *** VALIDATE ramfs *** [ 0.792105] *** VALIDATE hugetlbfs *** [ 0.793507] pnp: PnP ACPI init [ 0.794830] pnp 00:00: Plug and Play ACPI device, IDs PNP0303 (active) [ 0.794901] pnp 00:01: Plug and Play ACPI device, IDs PNP0f13 (active) [ 0.794927] pnp 00:02: [dma 2] [ 0.794985] pnp 00:02: Plug and Play ACPI device, IDs PNP0700 (active) [ 0.795047] pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active) [ 0.795100] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active) [ 0.795138] pnp 00:05: Plug and Play ACPI device, IDs PNP0b00 (active) [ 0.795725] pnp: PnP ACPI: found 6 devices [ 0.809278] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.812268] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.814052] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.815895] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.818545] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.820324] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.822918] NET: Registered protocol family 2 [ 0.825272] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.829749] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.833695] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.839107] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.842423] TCP: Hash tables configured (established 65536 bind 65536) [ 0.845219] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.847453] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.849922] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.853024] NET: Registered protocol family 1 [ 0.855677] RPC: Registered named UNIX socket transport module. [ 0.857982] RPC: Registered udp transport module. [ 0.859172] RPC: Registered tcp transport module. [ 0.860504] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.862489] NET: Registered protocol family 44 [ 0.863739] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.865351] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.866931] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.869232] PCI: CLS 0 bytes, default 64 [ 0.871762] Unpacking initramfs... [ 2.196212] debug: unmapping init [mem 0xffff8c203cc54000-0xffff8c203ffbffff] [ 2.199448] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.200961] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.203737] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.654276] Initialise system trusted keyrings [ 2.655974] Key type blacklist registered [ 2.657641] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.665832] zbud: loaded [ 2.668619] *** VALIDATE nfs *** [ 2.669784] *** VALIDATE nfs4 *** [ 2.671037] pstore: using deflate compression [ 2.674060] Platform Keyring initialized [ 2.771802] NET: Registered protocol family 38 [ 2.773730] Key type asymmetric registered [ 2.774762] Asymmetric key parser 'x509' registered [ 2.775974] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.778268] io scheduler mq-deadline registered [ 2.779880] io scheduler kyber registered [ 2.780996] io scheduler bfq registered [ 2.782211] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.784967] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.787225] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.789747] ACPI: Power Button [PWRF] [ 2.793837] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.799221] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.809816] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.816552] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.832462] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.860467] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.889562] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.894767] Non-volatile memory driver v1.3 [ 2.896133] Linux agpgart interface v0.103 [ 2.929542] virtio_blk virtio1: [vda] 145168 512-byte logical blocks (74.3 MB/70.9 MiB) [ 2.934473] vda: detected capacity change from 0 to 74326016 [ 2.948037] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.950349] vdb: detected capacity change from 0 to 1073741824 [ 2.962211] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.964961] vdc: detected capacity change from 0 to 2621440000 [ 2.976595] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.979105] vdd: detected capacity change from 0 to 2621440000 [ 2.990637] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.993288] vde: detected capacity change from 0 to 4294967296 [ 3.007783] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.010204] vdf: detected capacity change from 0 to 4294967296 [ 3.017131] libphy: Fixed MDIO Bus: probed [ 3.021859] usbcore: registered new interface driver usbserial_generic [ 3.024145] usbserial: USB Serial support registered for generic [ 3.026198] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.030050] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.031563] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.033599] mousedev: PS/2 mouse device common for all mice [ 3.036046] rtc_cmos 00:05: RTC can wake from S4 [ 3.038948] rtc_cmos 00:05: registered as rtc0 [ 3.039370] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.040796] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.046470] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.047106] intel_pstate: CPU model not supported [ 3.051971] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.056069] hid: raw HID events driver (C) Jiri Kosina [ 3.057977] usbcore: registered new interface driver usbhid [ 3.059943] usbhid: USB HID core driver [ 3.061320] drop_monitor: Initializing network drop monitor service [ 3.063617] Initializing XFRM netlink socket [ 3.065099] NET: Registered protocol family 10 [ 3.067835] Segment Routing with IPv6 [ 3.069046] NET: Registered protocol family 17 [ 3.070794] mpls_gso: MPLS GSO support [ 3.071918] start plist test [ 3.075073] end plist test [ 3.076547] RAS: Correctable Errors collector initialized. [ 3.079215] AVX version of gcm_enc/dec engaged. [ 3.080365] AES CTR mode by8 optimization enabled [ 3.137993] sched_clock: Marking stable (3137965777, 0)->(3903579204, -765613427) [ 3.141528] registered taskstats version 1 [ 3.143037] Loading compiled-in X.509 certificates [ 3.144857] zswap: loaded using pool lzo/zbud [ 3.162689] Key type big_key registered [ 3.173044] Key type encrypted registered [ 3.174658] ima: No TPM chip found, activating TPM-bypass! [ 3.176388] ima: Allocated hash algorithm: sha1 [ 3.177882] ima: No architecture policies found [ 3.179497] evm: Initialising EVM extended attributes: [ 3.181124] evm: security.selinux [ 3.182129] evm: security.ima [ 3.182834] evm: security.capability [ 3.183884] evm: HMAC attrs: 0x1 [ 3.186170] rtc_cmos 00:05: setting system clock to 2026-07-04 06:21:28 UTC (1783146088) [ 3.191570] debug: unmapping init [mem 0xffffffffa8803000-0xffffffffa89fffff] [ 3.194371] debug: unmapping init [mem 0xffffffffa7582000-0xffffffffa7858fff] [ 3.203083] Write protecting the kernel read-only data: 28672k [ 3.206317] debug: unmapping init [mem 0xffffffffa5c03000-0xffffffffa5dfffff] [ 3.208715] debug: unmapping init [mem 0xffffffffa6514000-0xffffffffa65fffff] [ 3.238740] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.248115] systemd[1]: Detected virtualization kvm. [ 3.250128] systemd[1]: Detected architecture x86-64. [ 3.252241] systemd[1]: Running in initial RAM disk. [ 3.278821] systemd[1]: No hostname configured. [ 3.280711] systemd[1]: Set hostname to . [ 3.283024] random: systemd: uninitialized urandom read (16 bytes read) [ 3.285739] systemd[1]: Initializing machine ID from random generator. [ 3.328391] random: ln: uninitialized urandom read (6 bytes read) [ 3.391550] random: systemd: uninitialized urandom read (16 bytes read) [ 3.394381] systemd[1]: Listening on udev Control Socket. [ 3.398340] systemd[1]: Reached target Swap. [ 3.402021] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ 3.972042] device-mapper: uevent: version 1.0.3 [ 3.973855] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ 4.628355] random: fast init done [ 4.639623] virtio_net virtio0 ens2: renamed from eth0 [ 4.642668] libata version 3.00 loaded. [ 4.645687] ata_piix 0000:00:01.1: version 2.13 [ 4.658829] scsi host0: ata_piix [ 4.679270] scsi host1: ata_piix [ 4.680937] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.683199] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.688391] random: crng init done [ 9.689668] random: 7 urandom warning(s) missed due to ratelimiting [ 9.911090] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ 10.992351] printk: systemd: 25 output lines suppressed due to ratelimiting [ 11.224027] SELinux: Disabled at runtime. [ 11.280592] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.287963] systemd[1]: Detected virtualization kvm. [ 11.289625] systemd[1]: Detected architecture x86-64. [ 11.704400] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.707688] systemd[1]: Stopped Switch Root. [ 11.711862] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.715482] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.718988] systemd[1]: Stopped Journal Service. [ 11.725979] systemd[1]: Starting Journal Service... [ 11.734530] systemd[1]: Activating swap /dev/disk/by-label/SWAP... [ 11.756757] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ 12.056742] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 12.323130] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.337929] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.464378] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.472419] EDAC sbridge: Seeking for: PCI ID 8086:0ea0 [ 12.472440] EDAC sbridge: Ver: 1.1.2 [ 14.073344] Key type dns_resolver registered [ 14.354321] NFS: Registering the id_resolver key type [ 14.355873] Key type id_resolver registered [ 14.357500] Key type id_legacy registered [ 35.509973] libcfs: loading out-of-tree module taints kernel. [ 35.526973] Key type ._llcrypt registered [ 35.528377] Key type .llcrypt registered [ 35.568336] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 43.086325] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 43.579141] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 43.585692] alg: No test for adler32 (adler32-zlib) [ 44.559065] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 44.837076] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 46.439155] Key type lgssc registered [ 47.068792] Lustre: Echo OBD driver; http://www.lustre.org/ [ 53.218572] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 66.506703] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 70.658535] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 70.665462] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 71.763706] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 71.774631] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 71.813551] Lustre: lustre-MDT0000: new disk, initializing [ 71.836276] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 71.842210] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 73.128095] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 78.465139] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 78.501349] Lustre: 6477:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 78.516128] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 78.518803] Lustre: Skipped 1 previous similar message [ 78.561444] Lustre: lustre-MDT0001: new disk, initializing [ 78.585109] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 78.596851] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 78.601487] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 79.962689] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 82.222896] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 85.284596] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 85.381110] Lustre: lustre-OST0000: new disk, initializing [ 85.383265] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 85.385735] Lustre: 8415:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 85.403582] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 87.338443] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 89.132956] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 89.136267] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 89.163389] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 92.928314] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 92.978585] Lustre: lustre-OST0001: new disk, initializing [ 92.980274] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 92.982589] Lustre: 9488:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 93.002295] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 94.929082] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 96.685047] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 96.688899] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 96.703937] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 100.756441] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 105.344329] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 107.223451] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing check_logdir /tmp/testlogs/ [ 108.658861] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing yml_node [ 110.023761] Lustre: DEBUG MARKER: Client: 2.17.54.83 [ 110.818810] Lustre: DEBUG MARKER: MDS: 2.17.54.83 [ 111.633413] Lustre: DEBUG MARKER: OSS: 2.17.54.83 [ 112.185056] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Jul 4 02:23:16 EDT 2026 [ 117.787504] Lustre: DEBUG MARKER: excepting tests: 32newtarball [ 118.322282] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 130.015572] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 130.019113] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 130.024118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 132.575506] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 132.576333] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 132.581927] Lustre: Skipped 1 previous similar message [ 137.695774] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 137.698433] Lustre: Skipped 6 previous similar messages [ 142.815847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 142.817837] Lustre: Skipped 3 previous similar messages [ 144.351140] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 144.409292] Lustre: server umount lustre-MDT0000 complete [ 145.647706] LustreError: 6470:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783146230 with bad export cookie 6588309134534833384 [ 145.648888] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 145.652456] LustreError: 6470:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 147.935738] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 147.936037] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 147.939097] Lustre: Skipped 2 previous similar messages [ 151.842331] Lustre: server umount lustre-MDT0001 complete [ 167.391136] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 167.429445] Lustre: server umount lustre-OST0000 complete [ 174.797108] Lustre: server umount lustre-OST0001 complete [ 177.014361] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 180.173369] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 192.340129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 192.436892] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 192.450576] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 192.487916] Lustre: lustre-MDT0000: new disk, initializing [ 192.512422] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 192.520721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 193.877419] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 198.117991] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 198.160934] Lustre: 15031:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 198.290489] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 199.738783] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 203.543613] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 205.426666] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 207.727768] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 207.841880] Lustre: lustre-OST0000: new disk, initializing [ 207.844422] Lustre: Skipped 1 previous similar message [ 207.847130] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 207.850048] Lustre: Skipped 2 previous similar messages [ 207.853104] Lustre: 17000:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 207.887659] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 207.890722] Lustre: Skipped 1 previous similar message [ 209.094592] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 209.098246] Lustre: Skipped 1 previous similar message [ 209.100997] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 209.150734] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 209.961352] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 213.759649] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 215.263308] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 215.348466] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 216.781178] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 216.849069] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 219.618216] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 219.618365] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 219.622911] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 219.625086] LustreError: Skipped 1 previous similar message [ 219.627631] Lustre: Skipped 3 previous similar messages [ 223.571532] Lustre: server umount lustre-OST0000 complete [ 229.344073] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 229.347314] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 229.353871] Lustre: Skipped 1 previous similar message [ 231.008779] Lustre: server umount lustre-MDT0000 complete [ 232.390119] LustreError: 15024:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783146317 with bad export cookie 6588309134534834903 [ 232.393310] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 232.395058] LustreError: 15024:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 232.527346] Lustre: server umount lustre-MDT0001 complete [ 234.489346] Lustre: DEBUG MARKER: == conf-sanity test 121: failover MGS ==================== 02:25:19 (1783146319) [ 240.936616] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 241.101393] LustreError: 19870:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 241.108544] LustreError: 19870:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 241.128184] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 242.506459] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 245.404300] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 246.995787] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 249.522982] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 251.465888] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 252.267276] Lustre: Failing over lustre-MDT0000 [ 252.357141] Lustre: server umount lustre-MDT0000 complete [ 255.328817] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 255.329420] LustreError: 19896:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 255.332625] Lustre: Skipped 2 previous similar messages [ 260.575972] LustreError: 19871:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 260.581944] LustreError: 19871:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 265.556644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 265.600364] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 267.181184] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 269.792640] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects [ 269.795610] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 269.801748] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted. [ 270.990689] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mgc.*.mgs_server_uuid [ 274.912491] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 274.912694] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 274.913334] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 274.913340] Lustre: Skipped 2 previous similar messages [ 274.918210] Lustre: Skipped 1 previous similar message [ 279.124303] Lustre: server umount lustre-MDT0000 complete [ 280.032097] LustreError: 19871:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 280.037867] LustreError: 19871:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 280.420370] LustreError: 19897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783146365 with bad export cookie 6588309134534836317 [ 280.423407] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 280.425600] LustreError: 19897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 280.549767] Lustre: server umount lustre-MDT0001 complete [ 283.686799] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 283.908026] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 283.910496] Lustre: Skipped 2 previous similar messages [ 285.351779] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 288.205043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 289.738107] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 292.094658] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 293.956598] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 294.735875] Lustre: Failing over lustre-MDT0000 [ 294.828274] Lustre: server umount lustre-MDT0000 complete [ 297.825387] LustreError: 23201:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 297.831385] LustreError: 23201:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 307.752638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 307.789772] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 309.224374] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 312.993516] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects [ 312.996384] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 312.999524] Lustre: Skipped 1 previous similar message [ 313.006247] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted. [ 313.015940] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mgc.*.mgs_server_uuid [ 315.015222] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 315.015847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 315.019503] Lustre: Skipped 3 previous similar messages [ 315.024105] Lustre: Skipped 2 previous similar messages [ 321.178361] Lustre: server umount lustre-MDT0000 complete [ 322.433357] LustreError: 23935:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783146407 with bad export cookie 6588309134534837255 [ 322.436883] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 322.438447] LustreError: 23935:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 322.550601] Lustre: server umount lustre-MDT0001 complete [ 325.567228] Lustre: DEBUG MARKER: == conf-sanity test 122a: Check OST sequence update ====== 02:26:50 (1783146410) [ 331.160198] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 334.506338] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 347.096481] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 350.765581] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 350.880667] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 350.892732] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 350.931093] Lustre: lustre-MDT0000: new disk, initializing [ 350.957979] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 350.960154] Lustre: Skipped 2 previous similar messages [ 350.966239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 352.360442] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 356.724245] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 356.770826] Lustre: 29566:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 356.896909] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 358.337451] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 362.021949] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 363.884948] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 366.182477] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 366.314284] Lustre: 31532:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 367.744720] Lustre: *** cfs_fail_loc=1e0, val=0*** [ 367.767150] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 367.774076] Lustre: lustre-OST0000: MDS LAST_ID [0x280000401:0x21:0x0] (33) is 65248 behind OST LAST_ID [0x280000401:0xff01:0x0] (65281), trust the OST [ 368.302973] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 372.071059] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 374.413517] Lustre: lustre-OST0000-osc-MDT0001: update sequence from 0x280000400 to 0x280000402 [ 380.777960] Lustre: server umount lustre-OST0000 complete [ 382.431891] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 382.434767] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 382.439595] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 382.442201] Lustre: Skipped 2 previous similar messages [ 392.701580] Lustre: server umount lustre-MDT0000 complete [ 393.696054] LustreError: 29577:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 393.702283] LustreError: 29577:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 393.877232] LustreError: 30360:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783146479 with bad export cookie 6588309134534837661 [ 393.880737] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 393.881034] LustreError: 30360:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 404.504346] Lustre: server umount lustre-MDT0001 complete [ 408.110122] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 409.128719] Key type lgssc unregistered [ 409.257362] LNet: 33484:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 409.260516] LNetError: 33484:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 409.268275] LNet: Removed LNI 192.168.204.105@tcp [ 409.613118] Key type .llcrypt unregistered [ 409.614727] Key type ._llcrypt unregistered [ 418.590733] Key type ._llcrypt registered [ 418.592327] Key type .llcrypt registered [ 418.627175] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 419.020151] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 419.028056] alg: No test for adler32 (adler32-zlib) [ 419.861991] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 419.940100] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 421.519107] Key type lgssc registered [ 421.834515] Lustre: Echo OBD driver; http://www.lustre.org/ [ 424.197184] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 424.202976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 425.340902] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 426.544306] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 429.090686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 429.191250] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 430.386886] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 432.605153] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 434.365508] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 436.470780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 436.581330] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 438.356303] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 439.656635] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000402:747 to 0x280000402:833) [ 440.543726] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 455.382538] Lustre: DEBUG MARKER: == conf-sanity test 123ab: llog_print params output values from set_param -P ========================================================== 02:29:00 (1783146540) [ 455.956309] Lustre: Modifying parameter general.jobid_name=TESTNAME in log params [ 456.503552] Lustre: Modifying parameter general.jobid_name=jobid_name=%e.root in log params [ 458.523344] Lustre: DEBUG MARKER: == conf-sanity test 123ac: llog_print with --start and --end ========================================================== 02:29:03 (1783146543) [ 461.140483] Lustre: DEBUG MARKER: == conf-sanity test 123ad: llog_print shows all records == 02:29:05 (1783146545) [ 461.478583] Lustre: Setting parameter lustre-OST0000-osc.osc.max_dirty_mb=462 in log lustre-client [ 464.132272] Lustre: Disabling parameter lustre-OST0000-osc.osc.max_dirty_mb= in log lustre-client [ 464.134738] Lustre: Skipped 2 previous similar messages [ 464.649270] Lustre: DEBUG MARKER: == conf-sanity test 123ae: llog_cancel can cancel requested record ========================================================== 02:29:09 (1783146549) [ 471.218624] Lustre: Disabling parameter lustre-OST0000-osc.osc.max_pages_per_rpc= in log lustre-client [ 471.221333] Lustre: Skipped 10 previous similar messages [ 472.078814] Lustre: DEBUG MARKER: == conf-sanity test 123af: llog_catlist can show all config files correctly ========================================================== 02:29:16 (1783146556) [ 472.962919] Lustre: *** cfs_fail_loc=131b, val=2*** [ 474.080261] Lustre: *** cfs_fail_loc=131b, val=2*** [ 476.779225] Lustre: DEBUG MARKER: == conf-sanity test 123ag: llog_print skips values deleted by set_param -P -d ========================================================== 02:29:21 (1783146561) [ 480.892441] Lustre: Setting parameter general.jobid_name=jobid_name=jobid_name=%e.root in log params [ 480.894825] Lustre: Skipped 7 previous similar messages [ 481.450201] Lustre: DEBUG MARKER: == conf-sanity test 123ah: del_ost cancels config log entries correctly ========================================================== 02:29:26 (1783146566) [ 485.857117] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 485.857241] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 485.859710] LustreError: Skipped 1 previous similar message [ 485.864280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 485.867813] Lustre: Skipped 1 previous similar message [ 490.977115] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 490.979645] Lustre: Skipped 1 previous similar message [ 491.213771] Lustre: server umount lustre-OST0000 complete [ 496.096753] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 496.097046] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 496.099733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 496.103428] Lustre: Skipped 2 previous similar messages [ 498.583983] Lustre: server umount lustre-MDT0000 complete [ 499.715485] LustreError: 34855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783146585 with bad export cookie 13172148689432941376 [ 499.717851] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 499.719260] LustreError: 34855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 499.822633] Lustre: server umount lustre-MDT0001 complete [ 503.229908] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 504.155795] Key type lgssc unregistered [ 504.282267] LNet: 42719:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 504.285046] LNetError: 42719:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 504.293254] LNet: Removed LNI 192.168.204.105@tcp [ 504.568205] Key type .llcrypt unregistered [ 504.569331] Key type ._llcrypt unregistered [ 510.264078] Key type ._llcrypt registered [ 510.265621] Key type .llcrypt registered [ 510.303499] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 516.299065] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 516.592733] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 516.629189] alg: No test for adler32 (adler32-zlib) [ 517.479686] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 517.561472] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 519.143141] Key type lgssc registered [ 519.497832] Lustre: Echo OBD driver; http://www.lustre.org/ [ 530.537961] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 530.544096] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 531.626285] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 531.637836] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 531.671735] Lustre: lustre-MDT0000: new disk, initializing [ 531.712150] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 531.718685] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 533.075866] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 537.000549] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 537.043443] Lustre: 45845:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 537.066584] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 537.069277] Lustre: Skipped 1 previous similar message [ 537.104900] Lustre: lustre-MDT0001: new disk, initializing [ 537.134650] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 537.145615] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 537.149463] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 538.384179] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 541.976955] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 543.604847] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 545.744820] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 545.841340] Lustre: lustre-OST0000: new disk, initializing [ 545.842934] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 545.845043] Lustre: 47811:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 545.869184] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 546.740975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 546.744821] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 546.771687] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 547.656540] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 551.218467] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 552.675542] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 552.742539] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 554.121093] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 554.197361] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 557.026401] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 557.026407] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 557.027189] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 557.029968] Lustre: Skipped 1 previous similar message [ 562.145594] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 562.147467] Lustre: Skipped 2 previous similar messages [ 567.267196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 567.269097] Lustre: Skipped 1 previous similar message [ 569.311162] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 569.370098] Lustre: server umount lustre-OST0000 complete [ 572.384581] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 572.385183] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 572.389174] Lustre: Skipped 1 previous similar message [ 576.726971] Lustre: server umount lustre-MDT0000 complete [ 577.934858] LustreError: 45836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783146663 with bad export cookie 7505621802021977788 [ 577.937495] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 577.939370] LustreError: 45836:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 578.046147] Lustre: server umount lustre-MDT0001 complete [ 581.254905] Lustre: DEBUG MARKER: == conf-sanity test 123ai: llog_print display all non skipped records ========================================================== 02:31:06 (1783146666) [ 583.699367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 583.859078] LustreError: 50268:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 583.866242] LustreError: 50268:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 583.887951] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 585.343402] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 588.309937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 588.442274] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 589.787520] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 592.193510] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 594.020621] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 596.141661] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 598.005724] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 600.178223] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 612.446340] LustreError: 52815:0:(llog.c:444:llog_init_handle()) llog type is not specified! [ 612.732121] Lustre: Modifying parameter general.timeout=1 in log params [ 613.793059] Lustre: Modifying parameter general.timeout=5 in log params [ 613.795128] Lustre: Skipped 3 previous similar messages [ 615.996983] Lustre: Modifying parameter general.timeout=13 in log params [ 615.998636] Lustre: Skipped 7 previous similar messages [ 620.222846] Lustre: Modifying parameter general.timeout=28 in log params [ 620.225147] Lustre: Skipped 14 previous similar messages [ 628.246235] Lustre: Modifying parameter general.timeout=57 in log params [ 628.248613] Lustre: Skipped 28 previous similar messages [ 644.540365] Lustre: Modifying parameter general.timeout=114 in log params [ 644.542526] Lustre: Skipped 56 previous similar messages [ 652.505731] Lustre: DEBUG MARKER: == conf-sanity test 123aj: check permanent TBF rules ===== 02:32:17 (1783146737) [ 653.447410] Lustre: 59713:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/ost.OSS.ost_io.nrs_tbf_rule=start rule4 jobid={fio.0.0} rate=500 (mode = 2) failed: rc = -17 [ 654.622843] Lustre: 59909:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/ost.OSS.ost_io.nrs_tbf_rule= start rule3 jobid={cat.0.0} rate=1000 (mode = 2) failed: rc = -17 [ 654.627836] Lustre: 59909:0:(mgs_llog.c:1447:mgs_modify_param()) Skipped 1 previous similar message [ 665.036393] Lustre: 61548:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/ost.OSS.ost_io.nrs_tbf_rule= (mode = 1) failed: rc = -2 [ 665.041831] Lustre: 61548:0:(mgs_llog.c:1447:mgs_modify_param()) Skipped 3 previous similar messages [ 665.049841] LustreError: 61548:0:(mgs_handler.c:1206:mgs_iocontrol()) MGS: setparam err: rc = -2 [ 666.364615] Lustre: DEBUG MARKER: == conf-sanity test 123F: clear and reset all parameters using set_param -F ========================================================== 02:32:31 (1783146751) [ 670.179583] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 670.180913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 670.183918] Lustre: Skipped 1 previous similar message [ 670.186260] Lustre: Skipped 1 previous similar message [ 676.070229] Lustre: server umount lustre-MDT0000 complete [ 677.242191] LustreError: 50250:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783146762 with bad export cookie 7505621802021978656 [ 677.244413] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 677.245942] LustreError: 50250:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 677.360923] Lustre: server umount lustre-MDT0001 complete [ 687.832147] Lustre: server umount lustre-OST0000 complete [ 696.170218] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 696.265479] Lustre: MGS: Logs for fs lustre were removed by user request. All servers must be restarted in order to regenerate the logs: rc = 0 [ 696.275749] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 696.279504] Lustre: Skipped 38 previous similar messages [ 696.360245] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 696.362012] Lustre: Skipped 1 previous similar message [ 697.619221] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 700.323678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 700.355932] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 700.364458] Lustre: 64070:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 701.862020] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 704.290798] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 706.033420] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 708.121625] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 708.184082] Lustre: MGS: Regenerating lustre-OST0000 log by user request: rc = 0 [ 709.997251] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 712.195135] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 712.697726] Lustre: 66634:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/jobid_var=TESTNAME (mode = 0) failed: rc = -17 [ 716.258230] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 716.259600] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 716.261704] LustreError: Skipped 2 previous similar messages [ 716.265791] Lustre: Skipped 1 previous similar message [ 716.266534] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 716.271473] Lustre: Skipped 6 previous similar messages [ 720.648846] Lustre: server umount lustre-OST0000 complete [ 725.983720] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 725.986112] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 725.991541] Lustre: Skipped 1 previous similar message [ 735.712229] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 735.714200] Lustre: Skipped 6 previous similar messages [ 736.223085] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 736.279923] Lustre: server umount lustre-MDT0000 complete [ 737.404055] LustreError: 66636:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783146822 with bad export cookie 7505621802021980945 [ 737.409642] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 737.504184] Lustre: server umount lustre-MDT0001 complete [ 740.955356] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 741.846585] Key type lgssc unregistered [ 741.965303] LNet: 67839:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 741.968237] LNetError: 67839:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 741.978268] LNet: Removed LNI 192.168.204.105@tcp [ 742.250496] Key type .llcrypt unregistered [ 742.251567] Key type ._llcrypt unregistered [ 751.200209] Key type ._llcrypt registered [ 751.201677] Key type .llcrypt registered [ 751.238798] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 751.599294] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 751.631060] alg: No test for adler32 (adler32-zlib) [ 752.495403] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 752.593965] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 754.183103] Key type lgssc registered [ 754.548357] Lustre: Echo OBD driver; http://www.lustre.org/ [ 757.271993] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 757.277952] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 758.425271] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 759.857316] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 762.615037] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 762.798374] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 764.119347] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 766.364724] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 768.046469] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 770.192731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 770.362561] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 772.116130] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 774.254968] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 787.482039] Lustre: Modifying parameter general.jobid_var=TESTNAME in log params [ 793.568159] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 793.572385] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 793.577072] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 794.082712] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 794.082792] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 794.088250] Lustre: Skipped 2 previous similar messages [ 796.121832] Lustre: server umount lustre-MDT0000 complete [ 797.332703] LustreError: 69208:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783146882 with bad export cookie 15116370765631958322 [ 797.335501] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 797.337263] LustreError: 69208:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 797.443501] Lustre: server umount lustre-MDT0001 complete [ 808.991837] Lustre: server umount lustre-OST0000 complete [ 817.779461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 817.871291] Lustre: MGS: Logs for fs lustre were removed by user request. All servers must be restarted in order to regenerate the logs: rc = 0 [ 817.881278] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 817.968469] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 819.250767] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 822.027437] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 822.055766] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 822.065299] Lustre: 74115:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 822.227093] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 823.546091] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 825.900681] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 827.581689] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 829.722755] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 829.788673] Lustre: MGS: Regenerating lustre-OST0000 log by user request: rc = 0 [ 831.841194] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 834.175489] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 839.872340] Lustre: 76686:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0000/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 840.472208] Lustre: Modifying parameter general.jobid_var=disable in log params [ 845.283468] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 845.284227] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 845.289345] Lustre: Skipped 1 previous similar message [ 845.294660] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 845.297684] Lustre: Skipped 1 previous similar message [ 847.760662] Lustre: server umount lustre-OST0000 complete [ 850.401616] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 850.402562] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 850.406810] Lustre: Skipped 1 previous similar message [ 855.191696] Lustre: server umount lustre-MDT0000 complete [ 855.519972] LustreError: 74125:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 855.526891] LustreError: 74125:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 856.419958] LustreError: 76693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783146941 with bad export cookie 15116370765631959141 [ 856.423499] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 856.424707] LustreError: 76693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 856.537649] Lustre: server umount lustre-MDT0001 complete [ 860.039451] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 861.022196] Key type lgssc unregistered [ 861.145295] LNet: 77894:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 861.147472] LNetError: 77894:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 861.155298] LNet: Removed LNI 192.168.204.105@tcp [ 861.450440] Key type .llcrypt unregistered [ 861.451484] Key type ._llcrypt unregistered [ 870.538807] Key type ._llcrypt registered [ 870.539802] Key type .llcrypt registered [ 870.574439] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 871.013832] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 871.035120] alg: No test for adler32 (adler32-zlib) [ 871.895764] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 871.994098] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 873.583148] Key type lgssc registered [ 873.955159] Lustre: Echo OBD driver; http://www.lustre.org/ [ 876.557345] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 876.563404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 877.711167] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 879.043078] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 881.741291] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 881.946277] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 883.295911] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 885.673101] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 887.560816] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 889.805161] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 889.994276] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 891.914338] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 894.202193] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 906.658453] Lustre: Modifying parameter general.jobid_var=TEST_123H in log params [ 907.247948] Lustre: Modifying parameter general.jobid_var=TEST_123H in log params [ 907.250220] Lustre: Skipped 1 previous similar message [ 908.437043] Lustre: Modifying parameter general.jobid_var=TEST_123H in log params [ 908.439899] Lustre: Skipped 3 previous similar messages [ 910.541545] Lustre: Modifying parameter general.jobid_var=jobid_var=disable in log params [ 910.543547] Lustre: Skipped 6 previous similar messages [ 914.782508] Lustre: Modifying parameter general.jobid_var=jobid_var=disable in log params [ 914.785116] Lustre: Skipped 13 previous similar messages [ 922.836955] Lustre: Modifying parameter general.jobid_var=TEST_123H in log params [ 922.839931] Lustre: Skipped 26 previous similar messages [ 939.025284] Lustre: 87376:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/jobid_var=jobid_var=disable (mode = 0) failed: rc = -17 [ 939.353412] Lustre: Disabling parameter general.jobid_var= in log params [ 939.356100] Lustre: Skipped 45 previous similar messages [ 939.873587] Lustre: DEBUG MARKER: == conf-sanity test 124: check failover after replace_nids ========================================================== 02:37:04 (1783147024) [ 940.406495] Lustre: DEBUG MARKER: SKIP: conf-sanity test_124 needs MDT failover setup [ 940.954692] Lustre: DEBUG MARKER: == conf-sanity test 126: mount in parallel shouldn't cause a crash ========================================================== 02:37:05 (1783147025) [ 943.584139] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 943.587029] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 943.591475] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 944.610875] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 944.613887] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 944.619437] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 949.729880] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 949.731597] Lustre: Skipped 2 previous similar messages [ 954.850323] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 954.852630] Lustre: Skipped 1 previous similar message [ 956.383089] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 956.442626] Lustre: server umount lustre-OST0000 complete [ 958.944131] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 958.946649] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 958.950732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 963.871573] Lustre: server umount lustre-MDT0000 complete [ 965.088369] LustreError: 79291:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 965.093344] LustreError: 79291:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 965.162648] LustreError: 79269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783147050 with bad export cookie 1688098101249626407 [ 965.167910] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 965.299368] Lustre: server umount lustre-MDT0001 complete [ 968.996532] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 970.114834] Key type lgssc unregistered [ 970.248436] LNet: 88625:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 970.252265] LNetError: 88625:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 970.262339] LNet: Removed LNI 192.168.204.105@tcp [ 970.597400] Key type .llcrypt unregistered [ 970.598569] Key type ._llcrypt unregistered [ 972.229924] Key type ._llcrypt registered [ 972.231566] Key type .llcrypt registered [ 972.273043] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_module ../libcfs/libcfs/libcfs [ 973.749418] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules [ 974.127556] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 974.173285] alg: No test for adler32 (adler32-zlib) [ 975.026563] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 975.121615] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 976.703129] Key type lgssc registered [ 977.118192] Lustre: Echo OBD driver; http://www.lustre.org/ [ 977.526105] LustreError: 89778:0:(lu_object.c:1482:lu_context_key_register()) cfs_race id 60d sleeping [ 984.965567] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 988.042420] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 998.699702] LustreError: 89778:0:(lu_object.c:1482:lu_context_key_register()) cfs_fail_race id 60d awake: rc=0 [ 998.707930] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 999.879106] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1001.289766] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1004.179560] Lustre: DEBUG MARKER: == conf-sanity test 127: direct io overwrite on full ost ========================================================== 02:38:08 (1783147088) [ 1005.709053] LustreError: 90834:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1005.714184] LustreError: 90834:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1005.717726] LustreError: 90834:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -5 [ 1005.838243] Lustre: server umount lustre-MDT0000 complete [ 1010.202705] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing unload_modules_local [ 1011.192287] Key type lgssc unregistered [ 1011.312320] LNet: 92326:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1011.315789] LNetError: 92326:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1011.326339] LNet: Removed LNI 192.168.204.105@tcp [ 1011.641034] Key type .llcrypt unregistered [ 1011.642685] Key type ._llcrypt unregistered [ 1018.760215] Key type ._llcrypt registered [ 1018.761470] Key type .llcrypt registered [ 1018.798818] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1019.105382] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1019.112410] alg: No test for adler32 (adler32-zlib) [ 1019.950485] Lustre: Lustre: Build Version: 2.17.54_83_gdf67fd6 [ 1020.029408] LNet: Added LNI 192.168.204.105@tcp [8/256/0/180] [ 1021.607114] Key type lgssc registered [ 1021.956279] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1024.397164] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1024.403371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1025.555779] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1026.855101] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1029.642827] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1029.855921] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1031.149511] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1033.615316] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1035.521259] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1037.821165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1038.052519] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1039.986777] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1042.374546] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1074.615840] Lustre: DEBUG MARKER: == conf-sanity test 128: Force using remote logs with --nolocallogs ========================================================== 02:39:19 (1783147159) [ 1075.205414] Lustre: DEBUG MARKER: SKIP: conf-sanity test_128 need separate mgs device [ 1075.840589] Lustre: DEBUG MARKER: == conf-sanity test 129: attempt to connect an OST with the same index should fail ========================================================== 02:39:20 (1783147160) [ 1081.314446] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1081.315111] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1081.315892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1081.324129] Lustre: Skipped 1 previous similar message [ 1083.038123] Lustre: server umount lustre-MDT0000 complete [ 1084.278296] LustreError: 93501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783147169 with bad export cookie 7942817701896007622 [ 1084.282206] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1084.285251] LustreError: 93501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1084.406543] Lustre: server umount lustre-MDT0001 complete [ 1096.063556] Lustre: server umount lustre-OST0000 complete [ 1099.781601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1099.929093] LustreError: 97808:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1099.934669] LustreError: 97808:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1099.980422] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1101.266168] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1104.163679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1104.388300] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1105.835879] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1108.270623] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1110.121770] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1114.506884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1114.593267] LustreError: Server lustre-OST0000 requested index 0, but that index is already in use. Use --writeconf to force [ 1114.595029] LustreError: 99894:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1114.597743] LustreError: 97798:0:(mgs_handler.c:605:mgs_target_reg()) Failed to write lustre-OST0000 log (-98) [ 1114.613972] LustreError: lustre-OST0000: the MGS refuses to allow this server to start: rc = -98. Please see messages on the MGS. [ 1114.617147] LustreError: 99888:0:(tgt_mount.c:2611:server_fill_super()) Unable to start targets: -98 [ 1114.619413] LustreError: 99888:0:(tgt_mount.c:2129:server_put_super()) no obd lustre-OST0000 [ 1114.621107] LustreError: 99888:0:(tgt_mount.c:116:server_deregister_mount()) lustre-OST0000 not registered [ 1114.645413] Lustre: server umount lustre-OST0000 complete [ 1114.647195] LustreError: 99888:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -98 [ 1117.504445] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1117.569619] LustreError: Server lustre-OST0000 requested index 0, but that index is already in use. Use --writeconf to force [ 1117.574389] LustreError: 97797:0:(mgs_handler.c:605:mgs_target_reg()) Failed to write lustre-OST0000 log (-98) [ 1117.585891] LustreError: lustre-OST0000: the MGS refuses to allow this server to start: rc = -98. Please see messages on the MGS. [ 1117.590208] LustreError: 100355:0:(tgt_mount.c:2611:server_fill_super()) Unable to start targets: -98 [ 1117.592336] LustreError: 100355:0:(tgt_mount.c:2129:server_put_super()) no obd lustre-OST0000 [ 1117.594047] LustreError: 100355:0:(tgt_mount.c:116:server_deregister_mount()) lustre-OST0000 not registered [ 1117.616426] Lustre: server umount lustre-OST0000 complete [ 1117.618037] LustreError: 100355:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -98 [ 1120.817183] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1120.902270] Lustre: MGS: Regenerating lustre-OST0000 log by user request: rc = 0 [ 1120.905144] Lustre: Found index 0 for lustre-OST0000, updating log [ 1120.908413] Lustre: Client log for lustre-OST0000 was not updated; writeconf the MDT first to regenerate it. [ 1120.943208] Lustre: lustre-OST0000: new disk, initializing [ 1120.945303] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1120.948527] Lustre: 100886:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1121.053204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1123.144191] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1124.493448] Lustre: Failing over lustre-OST0000 [ 1124.552956] Lustre: server umount lustre-OST0000 complete [ 1129.951860] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1129.955424] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1129.960630] Lustre: Skipped 1 previous similar message [ 1129.962583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1129.964771] Lustre: Skipped 2 previous similar messages [ 1130.976058] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1130.976345] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1136.096247] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1136.099267] Lustre: Skipped 2 previous similar messages [ 1140.191178] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1140.193594] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1140.198294] Lustre: Skipped 1 previous similar message [ 1140.253630] Lustre: server umount lustre-MDT0000 complete [ 1141.550424] LustreError: 97789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783147226 with bad export cookie 7942817701896010352 [ 1141.553401] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1141.555098] LustreError: 97789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1141.693860] Lustre: server umount lustre-MDT0001 complete [ 1144.566400] Lustre: DEBUG MARKER: == conf-sanity test 130: re-register an MDT after writeconf ========================================================== 02:40:29 (1783147229) [ 1149.432120] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1152.756836] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1152.873353] LustreError: 103225:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1152.896634] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1154.141537] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1156.732832] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1158.124291] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1158.989518] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 1160.989873] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1163.004050] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1165.705517] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1165.739637] Lustre: MGS: Regenerating lustre-OST0001 log by user request: rc = 0 [ 1165.772321] Lustre: lustre-OST0001: new disk, initializing [ 1165.774343] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1165.777455] Lustre: 105541:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1166.120992] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1166.124203] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1166.160578] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1167.724603] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1170.812762] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1176.479776] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 1178.994299] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.5@tcp (stopping) [ 1181.664656] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 1181.667310] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1184.022365] Lustre: server umount lustre-MDT0001 complete [ 1184.105542] LustreError: 103221:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.5@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1187.235637] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1187.267905] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 1187.270139] Lustre: Found index 1 for lustre-MDT0001, updating log [ 1187.283894] Lustre: 103215:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1187.572672] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1187.575959] Lustre: Skipped 3 previous similar messages [ 1188.708850] LustreError: 107256:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0001 attach error -17 [ 1188.711390] LustreError: 107256:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0001: setup up failed: rc -17 [ 1188.714774] LustreError: 103218:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 1188.744597] LustreError: 107270:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.105@tcp: cfg command failed: rc = -17 [ 1188.748915] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 1188.843598] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1191.885441] Lustre: DEBUG MARKER: == conf-sanity test 131: MDT backup restore with project ID and inheritance flag ========================================================== 02:41:16 (1783147276) [ 1192.932638] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1197.296529] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1203.100411] Lustre: 109154:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1211.291517] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1212.560967] Lustre: 110285:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1258.975513] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1258.980368] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1258.985606] Lustre: Skipped 2 previous similar messages [ 1258.987952] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1258.990416] Lustre: Skipped 4 previous similar messages [ 1264.537228] Lustre: server umount lustre-MDT0000 complete [ 1264.608639] LustreError: 103963:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1264.615883] LustreError: 103963:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1265.777602] LustreError: 103207:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783147351 with bad export cookie 7942817701896011290 [ 1265.778958] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1265.782266] LustreError: 103207:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1265.892605] Lustre: server umount lustre-MDT0001 complete [ 1276.932078] Lustre: server umount lustre-OST0000 complete [ 1288.410277] Lustre: server umount lustre-OST0001 complete [ 1290.185676] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1293.313426] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1296.126199] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: (null) [ 1299.258071] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: (null) [ 1305.761973] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1309.356047] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1309.365528] Lustre: lustre-MDT0000: reset Object Index mappings [ 1309.514722] LustreError: 114207:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1309.538299] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1310.948735] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1313.808092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1313.816672] Lustre: lustre-MDT0001: reset Object Index mappings [ 1315.504049] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1316.481395] Lustre: 115355:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1318.694841] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1319.977466] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:349 to 0x280000401:386) [ 1320.792953] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1323.552653] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1324.842689] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:157 to 0x2c0000401:193) [ 1324.843213] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:199 to 0x280000400:225) [ 1324.843437] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:200 to 0x2c0000400:225) [ 1325.680800] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1328.817086] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1330.034525] Lustre: 117232:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1345.319863] Lustre: DEBUG MARKER: == conf-sanity test 132: hsm_actions processed after failover ========================================================== 02:43:50 (1783147430) [ 1375.712481] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1375.716046] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1375.721494] Lustre: Skipped 3 previous similar messages [ 1375.723840] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1375.726532] Lustre: Skipped 4 previous similar messages [ 1380.653843] Lustre: server umount lustre-MDT0000 complete [ 1381.347949] LustreError: 114204:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1381.352066] LustreError: 114204:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 1381.886515] LustreError: 116543:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783147467 with bad export cookie 7942817701896058015 [ 1381.891807] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1388.295305] Lustre: server umount lustre-MDT0001 complete [ 1395.655351] Lustre: server umount lustre-OST0000 complete [ 1403.101377] Lustre: server umount lustre-OST0001 complete [ 1405.183483] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 1408.270487] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1420.424396] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1420.518435] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1420.531393] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1420.576361] Lustre: lustre-MDT0000: new disk, initializing [ 1420.604826] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1421.938903] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1424.937206] Lustre: server umount lustre-MDT0000 complete [ 1428.067036] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1428.167140] Lustre: Found index 0 for lustre-MDT0000, updating log [ 1428.172327] Lustre: 121489:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0000/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1428.179402] Lustre: Setting parameter lustre-MDT0000.mdt.hsm_control=enabled in log lustre-MDT0000 [ 1429.657917] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1431.354449] Lustre: server umount lustre-MDT0000 complete [ 1434.327453] Lustre: DEBUG MARKER: == conf-sanity test 133: stripe QOS: free space balance in a pool ========================================================== 02:45:19 (1783147519) [ 1434.950834] Lustre: DEBUG MARKER: SKIP: conf-sanity test_133 needs >= 4 OSTs [ 1435.585046] Lustre: DEBUG MARKER: == conf-sanity test 134: check_iam works without faults == 02:45:20 (1783147520) [ 1441.364753] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1444.651698] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1444.780111] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1444.782447] Lustre: Skipped 5 previous similar messages [ 1446.145866] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1449.028143] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1449.069150] Lustre: 123535:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1449.093963] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1449.096941] Lustre: Skipped 1 previous similar message [ 1449.134802] Lustre: lustre-MDT0001: new disk, initializing [ 1449.185070] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1449.191715] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1450.589487] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1452.839550] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 1455.025717] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1455.136962] Lustre: lustre-OST0000: new disk, initializing [ 1455.139447] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1455.143165] Lustre: 125117:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1455.865494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1455.870300] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1455.905586] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1457.137983] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1461.185967] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1461.252385] Lustre: 125988:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1462.981838] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1462.987295] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1463.015254] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1463.155344] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1467.636953] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1468.861078] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 1586.386851] Lustre: 132360:0:(osd_internal.h:1444:osd_trans_exec_op()) lustre-MDT0000: opcode 9: before 516 < left 1024, rollback = 8 [ 1586.389405] Lustre: 132360:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 0/0/0, destroy: 0/0/0 [ 1586.391287] Lustre: 132360:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 1/1/0 [ 1586.393445] Lustre: 132360:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 0/0/0, punch: 0/0/0, quota 1/3/0 [ 1586.395552] Lustre: 132360:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 0/0/0, delete: 256/1024/0 [ 1586.397488] Lustre: 132360:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1667.058207] Lustre: DEBUG MARKER: == conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 02:49:11 (1783147751) [ 1679.839841] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1679.842571] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1679.846919] Lustre: Skipped 5 previous similar messages [ 1679.848838] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1679.850607] Lustre: Skipped 6 previous similar messages [ 1684.729877] Lustre: server umount lustre-MDT0000 complete [ 1685.950705] LustreError: 125116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783147771 with bad export cookie 7942817701896111768 [ 1685.954113] LustreError: 125116:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1685.955545] LustreError: MGC192.168.204.105@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1688.544244] LustreError: 123543:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1688.551081] LustreError: 123543:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1700.319085] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1700.436922] Lustre: server umount lustre-MDT0001 complete [ 1707.783650] Lustre: server umount lustre-OST0000 complete [ 1723.359102] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1723.421735] Lustre: server umount lustre-OST0001 complete [ 1725.546506] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_hostid [ 1728.851696] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing load_modules_local [ 1740.303539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1740.384770] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1740.394929] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1740.396557] Lustre: Skipped 1 previous similar message [ 1740.430813] Lustre: lustre-MDT0000: new disk, initializing [ 1740.432222] Lustre: Skipped 1 previous similar message [ 1740.452149] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1740.454300] Lustre: Skipped 3 previous similar messages [ 1740.458833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1741.775351] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1745.761067] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1745.804218] Lustre: 135373:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1745.910791] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1747.187955] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1750.666476] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1752.402064] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1754.583036] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1754.691602] Lustre: 137340:0:(osd_compat.c:1271:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1756.473059] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1756.475476] Lustre: Skipped 1 previous similar message [ 1756.477643] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1756.510276] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1756.574197] Lustre: DEBUG MARKER: oleg405-server.virtnet: executing set_default_debug -1 all [ 1760.113708] Lustre: DEBUG MARKER: oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1763.115505] Lustre: lustre-MDD0000: changelog on [ 1764.006846] Lustre: lustre-MDD0001: changelog on [ 2159.204450] hrtimer: interrupt took 4918337 ns [ 2188.284088] Lustre: DEBUG MARKER: conf-sanity test_135: @@@@@@ FAIL: creation thread is running. Is changelog reader stuck?