[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 521681443 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.003310] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009013] pid_max: default: 32768 minimum: 301 [ 0.010130] LSM: Security Framework initializing [ 0.011058] Yama: becoming mindful. [ 0.012038] SELinux: Initializing. [ 0.013078] *** VALIDATE selinux *** [ 0.021728] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027380] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028159] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029100] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030128] *** VALIDATE tmpfs *** [ 0.031485] *** VALIDATE proc *** [ 0.032296] *** VALIDATE cgroup *** [ 0.033015] *** VALIDATE cgroup2 *** [ 0.034286] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035150] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037031] Spectre V2 : User space: Vulnerable [ 0.038010] Speculative Store Bypass: Vulnerable [ 0.041627] debug: unmapping init [mem 0xffffffffb1459000-0xffffffffb1460fff] [ 0.043158] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044698] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045025] ... version: 2 [ 0.046015] ... bit width: 48 [ 0.047013] ... generic registers: 4 [ 0.048024] ... value mask: 0000ffffffffffff [ 0.049014] ... max period: 00007fffffffffff [ 0.050018] ... fixed-purpose events: 3 [ 0.051014] ... event mask: 000000070000000f [ 0.052404] rcu: Hierarchical SRCU implementation. [ 0.054604] smp: Bringing up secondary CPUs ... [ 0.055832] x86: Booting SMP configuration: [ 0.056029] .... node #0, CPUs: #1 #2 #3 [ 0.059254] smp: Brought up 1 node, 4 CPUs [ 0.061012] smpboot: Max logical packages: 1 [ 0.062020] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.127313] node 0 deferred pages initialised in 63ms [ 0.130192] devtmpfs: initialized [ 0.131437] x86/mm: Memory block size: 128MB [ 0.136287] gcov: version magic: 0x41383552 [ 0.137849] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.141074] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.143402] pinctrl core: initialized pinctrl subsystem [ 0.145166] [ 0.145794] ************************************************************* [ 0.148012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.150007] ** ** [ 0.151010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.154011] ** ** [ 0.156012] ** This means that this kernel is built to expose internal ** [ 0.159012] ** IOMMU data structures, which may compromise security on ** [ 0.161014] ** your system. ** [ 0.163014] ** ** [ 0.165048] ** If you see this message and you are not debugging the ** [ 0.167013] ** kernel, report this immediately to your vendor! ** [ 0.170012] ** ** [ 0.172011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.174012] ************************************************************* [ 0.177909] NET: Registered protocol family 16 [ 0.179433] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.181061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.183050] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.186006] cpuidle: using governor menu [ 0.187263] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.189333] PCI: Using configuration type 1 for base access [ 0.190111] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.199135] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.202104] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.206084] cryptd: max_cpu_qlen set to 1000 [ 0.209362] ACPI: Added _OSI(Module Device) [ 0.211015] ACPI: Added _OSI(Processor Device) [ 0.212012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.214014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.219218] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.224423] ACPI: Interpreter enabled [ 0.226064] ACPI: PM: (supports S0 S3 S4 S5) [ 0.227009] ACPI: Using IOAPIC for interrupt routing [ 0.229113] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.232661] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.241648] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.244051] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.246016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.248084] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.253389] acpiphp: Slot [2] registered [ 0.255121] acpiphp: Slot [5] registered [ 0.256112] acpiphp: Slot [6] registered [ 0.258106] acpiphp: Slot [7] registered [ 0.259078] acpiphp: Slot [8] registered [ 0.260012] acpiphp: Slot [9] registered [ 0.261064] acpiphp: Slot [10] registered [ 0.262098] acpiphp: Slot [3] registered [ 0.263063] acpiphp: Slot [4] registered [ 0.263864] acpiphp: Slot [11] registered [ 0.265065] acpiphp: Slot [12] registered [ 0.265956] acpiphp: Slot [13] registered [ 0.267071] acpiphp: Slot [14] registered [ 0.268008] acpiphp: Slot [15] registered [ 0.268892] acpiphp: Slot [16] registered [ 0.270061] acpiphp: Slot [17] registered [ 0.270933] acpiphp: Slot [18] registered [ 0.271050] acpiphp: Slot [19] registered [ 0.271930] acpiphp: Slot [20] registered [ 0.273131] acpiphp: Slot [21] registered [ 0.274100] acpiphp: Slot [22] registered [ 0.276099] acpiphp: Slot [23] registered [ 0.277105] acpiphp: Slot [24] registered [ 0.279091] acpiphp: Slot [25] registered [ 0.280092] acpiphp: Slot [26] registered [ 0.281089] acpiphp: Slot [27] registered [ 0.283086] acpiphp: Slot [28] registered [ 0.284111] acpiphp: Slot [29] registered [ 0.285074] acpiphp: Slot [30] registered [ 0.285977] acpiphp: Slot [31] registered [ 0.291090] PCI host bridge to bus 0000:00 [ 0.292017] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.294018] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.295018] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.297018] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.299019] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.300022] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.302138] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.304756] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.307091] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.314579] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.319039] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.321013] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.323015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.324015] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.326503] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.327611] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.330032] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.332594] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.338012] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.356016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.361904] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.368136] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.390018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.396011] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.411014] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.421859] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.430017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.437017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.465020] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.475040] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.483015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.488013] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.507016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.516967] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.526012] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.530014] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.552016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.565794] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.571017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.576022] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.593063] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.604422] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.611023] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.617018] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.632024] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.642119] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.645448] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.649418] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.652444] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.655301] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.661116] iommu: Default domain type: Passthrough [ 0.664424] SCSI subsystem initialized [ 0.666121] ACPI: bus type USB registered [ 0.667136] usbcore: registered new interface driver usbfs [ 0.670133] usbcore: registered new interface driver hub [ 0.675058] usbcore: registered new device driver usb [ 0.677202] pps_core: LinuxPPS API ver. 1 registered [ 0.679010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.682117] PTP clock support registered [ 0.685193] EDAC MC: Ver: 3.0.0 [ 0.687195] PCI: Using ACPI for IRQ routing [ 0.688000] NetLabel: Initializing [ 0.690012] NetLabel: domain hash size = 128 [ 0.691009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.693085] NetLabel: unlabeled traffic allowed by default [ 0.695070] vgaarb: loaded [ 0.696031] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.698012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.705151] clocksource: Switched to clocksource kvm-clock [ 0.816265] VFS: Disk quotas dquot_6.6.0 [ 0.817704] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.819459] *** VALIDATE ramfs *** [ 0.820356] *** VALIDATE hugetlbfs *** [ 0.821457] pnp: PnP ACPI init [ 0.823431] pnp: PnP ACPI: found 6 devices [ 0.838831] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.841463] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.842898] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.844123] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.845936] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.847737] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.849717] NET: Registered protocol family 2 [ 0.851292] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.854434] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.856751] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.860602] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.863570] TCP: Hash tables configured (established 65536 bind 65536) [ 0.865869] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.869102] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.870966] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.874187] NET: Registered protocol family 1 [ 0.876397] RPC: Registered named UNIX socket transport module. [ 0.878134] RPC: Registered udp transport module. [ 0.879443] RPC: Registered tcp transport module. [ 0.880431] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.881740] NET: Registered protocol family 44 [ 0.883027] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.884345] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.885609] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.887200] PCI: CLS 0 bytes, default 64 [ 0.888411] Unpacking initramfs... [ 2.260294] debug: unmapping init [mem 0xffff9da3fcc54000-0xffff9da3fffbffff] [ 2.263294] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.264837] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.267187] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.757600] Initialise system trusted keyrings [ 2.759320] Key type blacklist registered [ 2.761210] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.771049] zbud: loaded [ 2.776335] *** VALIDATE nfs *** [ 2.777669] *** VALIDATE nfs4 *** [ 2.779091] pstore: using deflate compression [ 2.781899] Platform Keyring initialized [ 2.885822] NET: Registered protocol family 38 [ 2.887354] Key type asymmetric registered [ 2.888429] Asymmetric key parser 'x509' registered [ 2.889932] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.893283] io scheduler mq-deadline registered [ 2.894513] io scheduler kyber registered [ 2.895948] io scheduler bfq registered [ 2.897576] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.900330] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.905791] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.908215] ACPI: Power Button [PWRF] [ 2.913921] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.921666] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.936174] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.944541] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.964380] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.993851] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.024230] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.029254] Non-volatile memory driver v1.3 [ 3.030925] Linux agpgart interface v0.103 [ 3.064471] virtio_blk virtio1: [vda] 149456 512-byte logical blocks (76.5 MB/73.0 MiB) [ 3.067528] vda: detected capacity change from 0 to 76521472 [ 3.087564] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.090641] vdb: detected capacity change from 0 to 1073741824 [ 3.109750] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.113382] vdc: detected capacity change from 0 to 2621440000 [ 3.130199] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.132333] vdd: detected capacity change from 0 to 2621440000 [ 3.148606] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.151781] vde: detected capacity change from 0 to 4294967296 [ 3.165202] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.167107] vdf: detected capacity change from 0 to 4294967296 [ 3.172614] libphy: Fixed MDIO Bus: probed [ 3.181618] usbcore: registered new interface driver usbserial_generic [ 3.183786] usbserial: USB Serial support registered for generic [ 3.186139] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.189783] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.191242] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.193426] mousedev: PS/2 mouse device common for all mice [ 3.196570] rtc_cmos 00:05: RTC can wake from S4 [ 3.198715] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.202233] rtc_cmos 00:05: registered as rtc0 [ 3.203071] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.204816] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.204853] intel_pstate: CPU model not supported [ 3.208607] hid: raw HID events driver (C) Jiri Kosina [ 3.214089] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.216417] usbcore: registered new interface driver usbhid [ 3.221244] usbhid: USB HID core driver [ 3.222267] drop_monitor: Initializing network drop monitor service [ 3.223645] Initializing XFRM netlink socket [ 3.224887] NET: Registered protocol family 10 [ 3.227272] Segment Routing with IPv6 [ 3.228338] NET: Registered protocol family 17 [ 3.230057] mpls_gso: MPLS GSO support [ 3.234802] RAS: Correctable Errors collector initialized. [ 3.236403] AVX version of gcm_enc/dec engaged. [ 3.237533] AES CTR mode by8 optimization enabled [ 3.312783] sched_clock: Marking stable (3312761059, 0)->(4298827978, -986066919) [ 3.315482] registered taskstats version 1 [ 3.316984] Loading compiled-in X.509 certificates [ 3.319338] zswap: loaded using pool lzo/zbud [ 3.344300] Key type big_key registered [ 3.356917] Key type encrypted registered [ 3.358616] ima: No TPM chip found, activating TPM-bypass! [ 3.360924] ima: Allocated hash algorithm: sha1 [ 3.362675] ima: No architecture policies found [ 3.364600] evm: Initialising EVM extended attributes: [ 3.366476] evm: security.selinux [ 3.367690] evm: security.ima [ 3.369200] evm: security.capability [ 3.370866] evm: HMAC attrs: 0x1 [ 3.373461] rtc_cmos 00:05: setting system clock to 2026-08-22 06:55:22 UTC (1787381722) [ 3.379811] debug: unmapping init [mem 0xffffffffb2403000-0xffffffffb25fffff] [ 3.382929] debug: unmapping init [mem 0xffffffffb1182000-0xffffffffb1458fff] [ 3.393131] Write protecting the kernel read-only data: 28672k [ 3.395380] debug: unmapping init [mem 0xffffffffaf803000-0xffffffffaf9fffff] [ 3.397601] debug: unmapping init [mem 0xffffffffb0114000-0xffffffffb01fffff] [ 3.430147] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.435666] systemd[1]: Detected virtualization kvm. [ 3.436974] systemd[1]: Detected architecture x86-64. [ 3.438390] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.467379] systemd[1]: No hostname configured. [ 3.469222] systemd[1]: Set hostname to . [ 3.471443] random: systemd: uninitialized urandom read (16 bytes read) [ 3.473840] systemd[1]: Initializing machine ID from random generator. [ 3.625420] random: systemd: uninitialized urandom read (16 bytes read) [ 3.627260] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.630817] random: systemd: uninitialized urandom read (16 bytes read) [ 3.633512] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.639108] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Journal Service... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Control Socket. Starting Setup Virtual Console... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Slices. Starting Apply Kernel Variables... [ OK ] Reached target Sockets. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ 3.773611] random: fast init done [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.327608] device-mapper: uevent: version 1.0.3 [ 4.330306] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.127862] virtio_net virtio0 ens2: renamed from eth0 [ 5.136263] scsi host0: ata_piix [ 5.188164] scsi host1: ata_piix [ 5.189084] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.190465] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.889572] dracut-initqueue[575]: RTNETLINK answers: File exists [ 9.880242] random: crng init done [ 9.881707] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.379088] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.661198] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.921591] SELinux: Disabled at runtime. [ 11.982035] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.995236] systemd[1]: Detected virtualization kvm. [ 11.997121] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.880713] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.909404] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.955400] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.973611] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.991282] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.027898] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.041289] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting POSIX Message Queue File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Paths. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Reached target Local Encrypted Volumes. Mounting Kernel Debug File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-getty.slice. Mounting Huge Pages File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Apply Kernel Variables... [ OK ] Created slice system-serial\x2dgetty.slice. Starting Create list of required st…ce nodes for the current kernel... [ 13.668628] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 14.402821] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 15.432214] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 15.645456] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 16.121466] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 16.203947] EDAC sbridge: Ver: 1.1.2 [ 20.041131] Key type dns_resolver registered [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit)[ 21.148033] NFS: Registering the id_resolver key type [ 21.152099] Key type id_resolver registered [ 21.157126] Key type id_legacy registered [*** ] A start job is running for Configur…-only root support (8s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Load/Save Random Seed. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Login Service. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg249-server login: [ 44.523637] hrtimer: interrupt took 15431048 ns [ 89.821463] libcfs: loading out-of-tree module taints kernel. [ 89.865571] Key type ._llcrypt registered [ 89.869921] Key type .llcrypt registered [ 89.993662] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 111.605269] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 113.613655] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 113.633942] alg: No test for adler32 (adler32-zlib) [ 115.133376] Lustre: Lustre: Build Version: 2.17.54_114_g76b937b [ 116.166112] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 117.927839] Key type lgssc registered [ 120.116117] Lustre: Echo OBD driver; http://www.lustre.org/ [ 140.824807] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 185.273761] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 197.614728] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 197.661858] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 198.938818] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 198.970621] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 199.060678] Lustre: lustre-MDT0000: new disk, initializing [ 199.126728] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 199.150604] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 203.126472] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 215.587343] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 215.686031] Lustre: 6500:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 215.703569] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 215.707987] Lustre: Skipped 1 previous similar message [ 215.789634] Lustre: lustre-MDT0001: new disk, initializing [ 215.869147] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 215.889108] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 215.897070] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 220.125116] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 225.235798] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 234.752514] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 234.921078] Lustre: lustre-OST0000: new disk, initializing [ 234.929103] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 234.944940] Lustre: 8437:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 235.010305] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 236.793887] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 236.802876] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 236.915331] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 240.646331] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 255.225856] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 255.358824] Lustre: lustre-OST0001: new disk, initializing [ 255.362105] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 255.365836] Lustre: 9511:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 255.428470] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 261.969334] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 262.692376] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 262.701325] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 262.744166] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 274.456736] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 285.841279] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 292.193162] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing check_logdir /tmp/testlogs/ [ 296.973409] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing yml_node [ 303.282886] Lustre: DEBUG MARKER: Client: 2.17.54.114 [ 305.732896] Lustre: DEBUG MARKER: MDS: 2.17.54.114 [ 308.212199] Lustre: DEBUG MARKER: OSS: 2.17.54.114 [ 309.644936] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity ============----- Sat Aug 22 03:00:27 EDT 2026 [ 326.812075] Lustre: DEBUG MARKER: - need MDS1_VERSION <= 2.16.61-1-g89cf292a8c2 (34682482 <= 34618625) for LU-18938, skip 360 [ 328.303754] Lustre: DEBUG MARKER: - need MDS1_VERSION < v2_14_55-100-g8a84c7f9c7 (34682482 < 34486116) for LU-14927, skip 0f [ 329.800762] Lustre: DEBUG MARKER: - need OST1_VERSION < v2_17_50-410-gbd8e7b9555 (34682482 < 34681754) for LU-12550, skip 216 [ 331.589835] Lustre: DEBUG MARKER: excepting tests: 56oc 42a 42c 42b 118c 118d 407 119i 216 817 411a [ 332.894290] Lustre: DEBUG MARKER: skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842 51c 51e 834 [ 334.547840] Lustre: DEBUG MARKER: === sanity: start setup 03:00:52 (1787382052) === [ 339.696969] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing check_config_client /mnt/lustre [ 360.760435] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 363.998749] Lustre: 13529:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 368.093122] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 373.445581] Lustre: DEBUG MARKER: === sanity: finish setup 03:01:31 (1787382091) === [ 375.911913] Lustre: DEBUG MARKER: == sanity test 0a: touch; rm ============================= 03:01:34 (1787382094) [ 382.817658] Lustre: DEBUG MARKER: == sanity test 0b: chmod 0755 /mnt/lustre ======================================================================================= 03:01:40 (1787382100) [ 389.787424] Lustre: DEBUG MARKER: == sanity test 0c: check import proc ===================== 03:01:47 (1787382107) [ 397.579988] Lustre: DEBUG MARKER: == sanity test 0d: check export proc ===================== 03:01:55 (1787382115) [ 410.243847] Lustre: DEBUG MARKER: == sanity test 0e: Enable DNE MDT balancing for mkdir in the ROOT ========================================================== 03:02:08 (1787382128) [ 417.465202] Lustre: DEBUG MARKER: == sanity test 1: mkdir; remkdir; rmdir ================== 03:02:15 (1787382135) [ 425.126760] Lustre: DEBUG MARKER: == sanity test 2: mkdir; touch; rmdir; check file ======== 03:02:23 (1787382143) [ 432.704312] Lustre: DEBUG MARKER: == sanity test 3: mkdir; touch; rmdir; check dir ========= 03:02:30 (1787382150) [ 440.266216] Lustre: DEBUG MARKER: == sanity test 4: mkdir; touch dir/file; rmdir; checkdir (expect error) ========================================================== 03:02:38 (1787382158) [ 448.352584] Lustre: DEBUG MARKER: == sanity test 5: mkdir .../d5 .../d5/d2; chmod .../d5/d2 ========================================================== 03:02:46 (1787382166) [ 455.524755] Lustre: DEBUG MARKER: == sanity test 6a: touch f6a; chmod f6a; runas -u 500 -g 500 chmod f6a (should return error) ============================================================ 03:02:53 (1787382173) [ 462.810773] Lustre: DEBUG MARKER: == sanity test 6c: touch f6c; chown f6c; runas -u 500 -g 500 chown f6c (should return error) ============================================================ 03:03:00 (1787382180) [ 470.062825] Lustre: DEBUG MARKER: == sanity test 6e: touch+chgrp ; runas -u 500 -g 500 chgrp (should return error) ========================================================== 03:03:08 (1787382188) [ 477.306941] Lustre: DEBUG MARKER: == sanity test 6g: verify new dir in sgid dir inherits group ========================================================== 03:03:15 (1787382195) [ 485.078323] Lustre: DEBUG MARKER: == sanity test 6h: runas -u 500 -g 500 chown RUNAS_ID.0 .../ (should return error) ========================================================== 03:03:23 (1787382203) [ 491.722811] Lustre: DEBUG MARKER: == sanity test 6i: touch+chmod+chgrp ; chgrp read-only file should succeed ========================================================== 03:03:30 (1787382210) [ 498.519513] Lustre: DEBUG MARKER: == sanity test 7a: mkdir .../d7; mcreate .../d7/f; chmod .../d7/f ============================================================== 03:03:36 (1787382216) [ 505.197586] Lustre: DEBUG MARKER: == sanity test 7b: mkdir .../d7; mcreate d7/f2; echo foo > d7/f2 =============================================================== 03:03:43 (1787382223) [ 512.531547] Lustre: DEBUG MARKER: == sanity test 8: mkdir .../d8; touch .../d8/f; chmod .../d8/f ================================================================= 03:03:50 (1787382230) [ 518.574408] Lustre: DEBUG MARKER: == sanity test 9: mkdir .../d9 .../d9/d2 .../d9/d2/d3 ========================================================================== 03:03:56 (1787382236) [ 525.654859] Lustre: DEBUG MARKER: == sanity test 10: mkdir .../d10 .../d10/d2; touch .../d10/d2/f ================================================================ 03:04:03 (1787382243) [ 533.774762] Lustre: DEBUG MARKER: == sanity test 11: mkdir .../d11 d11/d2; chmod .../d11/d2 ====================================================================== 03:04:11 (1787382251) [ 541.519719] Lustre: DEBUG MARKER: == sanity test 12: touch .../d12/f; chmod .../d12/f .../d12/f ================================================================== 03:04:19 (1787382259) [ 548.669874] Lustre: DEBUG MARKER: == sanity test 13: creat .../d13/f; dd .../d13/f; > .../d13/f ================================================================== 03:04:26 (1787382266) [ 557.480122] Lustre: DEBUG MARKER: == sanity test 14: touch .../d14/f; rm .../d14/f; rm .../d14/f ================================================================= 03:04:34 (1787382274) [ 564.930242] Lustre: DEBUG MARKER: == sanity test 15: touch .../d15/f; mv .../d15/f .../d15/f2 ==================================================================== 03:04:42 (1787382282) [ 572.770261] Lustre: DEBUG MARKER: == sanity test 16: touch .../d16/f; rm -rf .../d16/f ===== 03:04:50 (1787382290) [ 581.259059] Lustre: DEBUG MARKER: == sanity test 17a: symlinks: create, remove (real) ====== 03:04:58 (1787382298) [ 589.509995] Lustre: DEBUG MARKER: == sanity test 17b: symlinks: create, remove (dangling) == 03:05:07 (1787382307) [ 596.637410] Lustre: DEBUG MARKER: == sanity test 17c: symlinks: open dangling (should return error) ========================================================== 03:05:14 (1787382314) [ 603.503908] Lustre: DEBUG MARKER: == sanity test 17d: symlinks: create dangling ============ 03:05:21 (1787382321) [ 610.862143] Lustre: DEBUG MARKER: == sanity test 17e: symlinks: create recursive symlink (should return error) ========================================================== 03:05:28 (1787382328) [ 617.523536] Lustre: DEBUG MARKER: == sanity test 17f: symlinks: long and very long symlink name ========================================================== 03:05:35 (1787382335) [ 623.663336] Lustre: DEBUG MARKER: == sanity test 17g: symlinks: really long symlink name and inode boundaries ========================================================== 03:05:41 (1787382341) [ 630.905708] Lustre: DEBUG MARKER: == sanity test 17h: create objects: lov_free_memmd() doesn't lbug ========================================================== 03:05:49 (1787382349) [ 639.053664] Lustre: DEBUG MARKER: == sanity test 17i: don't panic on short symlink (should return error) ========================================================== 03:05:56 (1787382356) [ 640.506900] Lustre: *** cfs_fail_loc=143, val=0*** [ 647.059923] Lustre: DEBUG MARKER: == sanity test 17k: symlinks: rsync with xattrs enabled == 03:06:05 (1787382365) [ 654.983272] Lustre: DEBUG MARKER: == sanity test 17l: Ensure lgetxattr's returned xattr size is consistent ========================================================== 03:06:13 (1787382373) [ 662.634496] Lustre: DEBUG MARKER: == sanity test 17m: run e2fsck against MDT which contains short/long symlink ========================================================== 03:06:20 (1787382380) [ 735.575476] Lustre: Failing over lustre-MDT0001 [ 735.712857] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 735.720294] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 735.726952] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 735.828461] Lustre: server umount lustre-MDT0001 complete [ 738.787230] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 738.796741] Lustre: Skipped 1 previous similar message [ 743.921444] LustreError: 6514:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 743.944787] LustreError: 6514:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 745.015174] LustreError: 13553:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.49@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 745.676510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 745.917791] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 745.959419] Lustre: lustre-MDT0001: in recovery but waiting for the first client to connect [ 747.158872] Lustre: lustre-MDT0001: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 750.328677] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 751.092940] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 751.153743] Lustre: lustre-MDT0001: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 751.212911] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:7 to 0x280000400:33) [ 751.216751] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:6 to 0x2c0000400:33) [ 781.468061] Lustre: DEBUG MARKER: == sanity test 17n: run e2fsck against master/slave MDT which contains remote dir ========================================================== 03:08:19 (1787382499) [ 788.496329] Lustre: Failing over lustre-MDT0000 [ 789.129442] Lustre: server umount lustre-MDT0000 complete [ 792.033734] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 792.042899] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 792.064056] LustreError: 11761:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 792.066012] Lustre: Skipped 3 previous similar messages [ 792.086163] LustreError: 11761:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 797.181054] LustreError: 13553:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 797.223490] LustreError: 13553:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 802.135456] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 802.277287] LustreError: 6509:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 802.309433] LustreError: 6509:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 802.321626] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 802.845661] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 802.921881] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 803.364925] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 807.935501] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 807.950547] Lustre: Skipped 2 previous similar messages [ 808.014285] Lustre: lustre-MDT0000: Recovery over after 0:05, of 2 clients 2 recovered and 0 were evicted. [ 808.121858] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:46 to 0x280000401:65) [ 808.122550] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:65) [ 808.666613] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 813.108807] Lustre: Failing over lustre-MDT0001 [ 813.531749] Lustre: server umount lustre-MDT0001 complete [ 813.664590] LustreError: 6509:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.49@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 818.147541] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 818.178689] Lustre: Skipped 1 previous similar message [ 824.861987] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 825.284226] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 825.344928] Lustre: lustre-MDT0001: in recovery but waiting for the first client to connect [ 827.312111] Lustre: lustre-MDT0001: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 829.695615] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 830.468391] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 830.480666] Lustre: Skipped 3 previous similar messages [ 830.540216] Lustre: lustre-MDT0001: Recovery over after 0:03, of 2 clients 2 recovered and 0 were evicted. [ 830.613147] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:49 to 0x280000400:65) [ 830.615499] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:49 to 0x2c0000400:65) [ 840.647185] Lustre: Failing over lustre-MDT0000 [ 840.700291] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 840.720552] Lustre: Skipped 1 previous similar message [ 840.731500] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 840.735044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 840.756215] Lustre: Skipped 1 previous similar message [ 841.086627] Lustre: server umount lustre-MDT0000 complete [ 843.332513] LustreError: 6508:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.49@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 843.343212] LustreError: 6508:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 856.569085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 856.774650] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 857.046629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 857.103701] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 858.659254] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 862.002771] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 862.223486] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 862.235725] Lustre: Skipped 2 previous similar messages [ 862.315495] Lustre: lustre-MDT0000: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 862.383716] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:97) [ 862.386274] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:46 to 0x280000401:97) [ 866.048524] Lustre: Failing over lustre-MDT0001 [ 866.323113] Lustre: server umount lustre-MDT0001 complete [ 867.298749] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 867.310242] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 867.335866] Lustre: Skipped 5 previous similar messages [ 877.538289] LustreError: 6509:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 877.564738] LustreError: 6509:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 24 previous similar messages [ 877.591837] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 877.992921] Lustre: lustre-MDT0001: in recovery but waiting for the first client to connect [ 879.153367] Lustre: lustre-MDT0001: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 883.083153] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 883.197418] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 883.207733] Lustre: Skipped 3 previous similar messages [ 883.286331] Lustre: lustre-MDT0001: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 883.355837] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:49 to 0x280000400:97) [ 883.358544] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:49 to 0x2c0000400:97) [ 901.580518] Lustre: Failing over lustre-MDT0000 [ 901.943158] Lustre: server umount lustre-MDT0000 complete [ 903.652159] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 903.655801] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 903.673564] Lustre: Skipped 1 previous similar message [ 912.288799] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 912.420942] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 912.703875] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 912.708987] Lustre: Skipped 1 previous similar message [ 912.760556] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 914.991862] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 917.819572] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 918.035792] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 918.041165] Lustre: Skipped 2 previous similar messages [ 918.241145] Lustre: lustre-MDT0000: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 918.291261] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:123 to 0x280000401:161) [ 918.293190] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:123 to 0x2c0000401:161) [ 922.221149] Lustre: Failing over lustre-MDT0001 [ 922.526952] Lustre: server umount lustre-MDT0001 complete [ 923.122195] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 933.892573] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 934.373167] Lustre: lustre-MDT0001: in recovery but waiting for the first client to connect [ 935.477515] Lustre: lustre-MDT0001: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 939.499512] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 939.503884] Lustre: Skipped 3 previous similar messages [ 939.544209] Lustre: lustre-MDT0001: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 939.603750] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:123 to 0x280000400:161) [ 939.605024] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:123 to 0x2c0000400:161) [ 939.754651] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 948.077038] Lustre: Failing over lustre-MDT0000 [ 948.487474] Lustre: server umount lustre-MDT0000 complete [ 949.729462] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 949.734980] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 949.742414] LustreError: 11761:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 949.742425] LustreError: 11761:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 21 previous similar messages [ 949.787025] Lustre: Skipped 7 previous similar messages [ 959.207733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 959.296585] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 963.909803] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 964.956753] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:123 to 0x2c0000401:193) [ 964.957381] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:123 to 0x280000401:193) [ 968.259422] Lustre: Failing over lustre-MDT0001 [ 968.664140] Lustre: server umount lustre-MDT0001 complete [ 980.367738] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 980.693162] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 980.705285] Lustre: Skipped 2 previous similar messages [ 980.762181] Lustre: lustre-MDT0001: in recovery but waiting for the first client to connect [ 980.779650] Lustre: Skipped 1 previous similar message [ 981.539054] Lustre: lustre-MDT0001: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 981.551765] Lustre: Skipped 1 previous similar message [ 985.533429] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 986.104703] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 986.112628] Lustre: Skipped 6 previous similar messages [ 986.147500] Lustre: lustre-MDT0001: Recovery over after 0:05, of 2 clients 2 recovered and 0 were evicted. [ 986.158916] Lustre: Skipped 1 previous similar message [ 986.224557] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:123 to 0x280000400:193) [ 986.226326] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:123 to 0x2c0000400:193) [ 996.584870] Lustre: DEBUG MARKER: == sanity test 17o: stat file with incompat LMA feature == 03:11:54 (1787382714) [ 999.581903] Lustre: Failing over lustre-MDT0000 [ 1000.171625] Lustre: server umount lustre-MDT0000 complete [ 1000.931971] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1000.948232] LustreError: Skipped 1 previous similar message [ 1017.826096] Lustre: 3636:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787382720/real 1787382720] req@ffff9da47f8fa680 x1874205695857152/t0(0) o400->MGC192.168.202.149@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787382736 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1017.843461] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1020.248041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1027.104740] LustreError: 3634:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9da343536680 x1874205695865344/t0(0) o250->MGC192.168.202.149@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1033.153027] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:123 to 0x280000401:225) [ 1033.157333] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:195 to 0x2c0000401:225) [ 1033.273594] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 1044.743536] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1046.595926] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1049.604909] Lustre: *** cfs_fail_loc=194, val=0*** [ 1049.608744] Lustre: 6510:0:(osd_handler.c:873:osd_check_lma()) lustre-MDT0000: unsupported incompat LMA feature(s) 0x0 for fid = [0x200000402:0x3ac:0x0], ino = 512165 [ 1057.781759] Lustre: DEBUG MARKER: == sanity test 17p: symlink overwrite directory error message ========================================================== 03:12:55 (1787382775) [ 1065.394973] Lustre: DEBUG MARKER: == sanity test 17q: set large xattr on fast symlink ====== 03:13:03 (1787382783) [ 1069.554099] sysctl (33328): drop_caches: 3 [ 1078.133899] Lustre: DEBUG MARKER: == sanity test 18: touch .../f ; ls ... ======================================================================================== 03:13:16 (1787382796) [ 1085.573386] Lustre: DEBUG MARKER: == sanity test 19a: touch .../f19 ; ls -l ... ; rm .../f19 ===================================================================== 03:13:23 (1787382803) [ 1096.088945] Lustre: DEBUG MARKER: == sanity test 19b: ls -l .../f19 (should return error) ======================================================================== 03:13:33 (1787382813) [ 1102.739273] Lustre: DEBUG MARKER: == sanity test 19c: runas -u 500 -g 500 touch .../f19 (should return error) ============================================================ 03:13:40 (1787382820) [ 1112.022776] Lustre: DEBUG MARKER: == sanity test 19d: cat .../f19 (should return error) ======================================================================== 03:13:49 (1787382829) [ 1120.576861] Lustre: DEBUG MARKER: == sanity test 20: touch .../f ; ls -l ... =============== 03:13:57 (1787382837) [ 1127.383678] Lustre: DEBUG MARKER: == sanity test 21: write to dangling link ================ 03:14:05 (1787382845) [ 1135.681936] Lustre: DEBUG MARKER: == sanity test 22: unpack tar archive as non-root user === 03:14:13 (1787382853) [ 1143.781654] Lustre: DEBUG MARKER: == sanity test 23a: O_CREAT|O_EXCL in subdir ============= 03:14:22 (1787382862) [ 1151.768539] Lustre: DEBUG MARKER: == sanity test 23b: O_APPEND check ======================= 03:14:29 (1787382869) [ 1159.292617] Lustre: DEBUG MARKER: == sanity test 23c: O_APPEND size checks for tiny writes ========================================================== 03:14:37 (1787382877) [ 1177.402193] Lustre: DEBUG MARKER: == sanity test 23d: file offset is correct after appending writes ========================================================== 03:14:55 (1787382895) [ 1184.703265] Lustre: DEBUG MARKER: == sanity test 24a: rename file to non-existent target === 03:15:02 (1787382902) [ 1192.922173] Lustre: DEBUG MARKER: == sanity test 24b: rename file to existing target ======= 03:15:10 (1787382910) [ 1203.532583] Lustre: DEBUG MARKER: == sanity test 24c: rename directory to non-existent target ========================================================== 03:15:21 (1787382921) [ 1211.958732] Lustre: DEBUG MARKER: == sanity test 24d: rename directory to existing target == 03:15:30 (1787382930) [ 1218.965781] Lustre: DEBUG MARKER: == sanity test 24e: touch .../R5a/f; rename .../R5a/f .../R5b/g ================================================================ 03:15:37 (1787382937) [ 1226.379588] Lustre: DEBUG MARKER: == sanity test 24f: touch .../R6a/f R6b/g; mv .../R6a/f .../R6b/g ============================================================== 03:15:44 (1787382944) [ 1233.850284] Lustre: DEBUG MARKER: == sanity test 24g: mkdir .../R7a/d; .../R7b/d; mv .../R7a/d .../R7b/e ================================================================ 03:15:52 (1787382952) [ 1243.020754] Lustre: DEBUG MARKER: == sanity test 24h: mkdir .../R8a/d; .../R8a/e; .../R8b/d; .../R8b/e; rename .../R8a/d .../R8b/e ========================================================== 03:16:00 (1787382960) [ 1252.502616] Lustre: DEBUG MARKER: == sanity test 24i: rename file to dir error: touch f ; mkdir a ; rename f a ========================================================== 03:16:10 (1787382970) [ 1262.521922] Lustre: DEBUG MARKER: == sanity test 24j: source does not exist ====================================================================================== 03:16:20 (1787382980) [ 1270.797772] Lustre: DEBUG MARKER: == sanity test 24k: touch .../R11a/f; mv .../R11a/f .../R11a/d ================================================================= 03:16:28 (1787382988) [ 1278.917755] Lustre: DEBUG MARKER: == sanity test 24l: Renaming a file to itself ================================================================================== 03:16:36 (1787382996) [ 1284.776792] Lustre: DEBUG MARKER: == sanity test 24m: Renaming a file to a hard link to itself =================================================================== 03:16:42 (1787383002) [ 1291.387637] Lustre: DEBUG MARKER: == sanity test 24n: Statting the old file after renaming (Posix rename 2) ========================================================== 03:16:49 (1787383009) [ 1298.261139] Lustre: DEBUG MARKER: == sanity test 24o: rename of files during htree split === 03:16:56 (1787383016) [ 1998.477805] Lustre: DEBUG MARKER: == sanity test 24p: mkdir .../R12a; .../R12b; rename .../R12a .../R12b ========================================================== 03:28:37 (1787383717) [ 2005.703482] Lustre: DEBUG MARKER: == sanity test 24q: mkdir .../R13a; .../R13b; open R13b rename R13a R13b ============================================================= 03:28:43 (1787383723) [ 2013.829860] Lustre: DEBUG MARKER: == sanity test 24r: mkdir .../R14a/b; rename .../R14a .../R14a/b =============================================================== 03:28:52 (1787383732) [ 2022.438708] Lustre: DEBUG MARKER: == sanity test 24s: mkdir .../R15a/b/c; rename .../R15a .../R15a/b/c =========================================================== 03:29:00 (1787383740) [ 2030.975876] Lustre: DEBUG MARKER: == sanity test 24t: mkdir .../R16a/b/c; rename .../R16a/b/c .../R16a =========================================================== 03:29:08 (1787383748) [ 2038.136539] Lustre: DEBUG MARKER: == sanity test 24u: create stripe file =================== 03:29:16 (1787383756) [ 2046.141873] Lustre: DEBUG MARKER: == sanity test 24v: list large directory (test hash collision, b=17560) ========================================================== 03:29:24 (1787383764) [ 4015.740225] Lustre: DEBUG MARKER: == sanity test 24w: Reading a file larger than 4Gb ======= 04:02:13 (1787385733) [ 4025.238392] Lustre: DEBUG MARKER: == sanity test 24x: cross MDT rename/link ================ 04:02:22 (1787385742) [ 4037.586696] Lustre: DEBUG MARKER: == sanity test 24y: rename/link on the same dir should succeed ========================================================== 04:02:34 (1787385754) [ 4050.799836] Lustre: DEBUG MARKER: == sanity test 24z: cross-MDT rename is done as cp ======= 04:02:48 (1787385768) [ 4065.394088] Lustre: DEBUG MARKER: == sanity test 24A: readdir() returns correct number of entries. ========================================================== 04:03:03 (1787385783) [ 4188.881449] Lustre: DEBUG MARKER: == sanity test 24B: readdir for striped dir return correct number of entries ========================================================== 04:05:06 (1787385906) [ 4197.692064] Lustre: DEBUG MARKER: == sanity test 24C: check .. in striped dir ============== 04:05:15 (1787385915) [ 4205.876909] Lustre: DEBUG MARKER: == sanity test 24E: cross MDT rename/link ================ 04:05:23 (1787385923) [ 4207.356379] Lustre: DEBUG MARKER: SKIP: sanity test_24E needs >= 4 MDTs [ 4209.137691] Lustre: DEBUG MARKER: == sanity test 24F: hash order vs readdir (LU-11330) ===== 04:05:27 (1787385927) [ 4280.014920] Lustre: DEBUG MARKER: == sanity test 24G: migrate symlink in rename ============ 04:06:37 (1787385997) [ 4288.290564] Lustre: DEBUG MARKER: == sanity test 24H: repeat FLD_QUERY rpc ================= 04:06:46 (1787386006) [ 4298.429897] Lustre: DEBUG MARKER: == sanity test 24I: large striped mkdir with limit check ========================================================== 04:06:56 (1787386016) [ 4305.938971] Lustre: DEBUG MARKER: == sanity test 25a: create file in symlinked directory ========================================================================= 04:07:03 (1787386023) [ 4314.234297] Lustre: DEBUG MARKER: == sanity test 25b: lookup file in symlinked directory ========================================================================= 04:07:11 (1787386031) [ 4320.776703] Lustre: DEBUG MARKER: == sanity test 26a: multiple component symlink ================================================================================= 04:07:18 (1787386038) [ 4327.403605] Lustre: DEBUG MARKER: == sanity test 26b: multiple component symlink at end of lookup ================================================================ 04:07:25 (1787386045) [ 4335.806805] Lustre: DEBUG MARKER: == sanity test 26c: chain of symlinks ==================== 04:07:33 (1787386053) [ 4343.562433] Lustre: DEBUG MARKER: == sanity test 26d: create multiple component recursive symlink ========================================================== 04:07:41 (1787386061) [ 4352.008673] Lustre: DEBUG MARKER: == sanity test 26e: unlink multiple component recursive symlink ========================================================== 04:07:49 (1787386069) [ 4359.327664] Lustre: DEBUG MARKER: == sanity test 26f: rm -r of a directory which has recursive symlink ========================================================== 04:07:57 (1787386077) [ 4369.622858] Lustre: DEBUG MARKER: == sanity test 27a: one stripe file ====================== 04:08:07 (1787386087) [ 4377.498968] Lustre: DEBUG MARKER: == sanity test 27b: create and write to two stripe file == 04:08:15 (1787386095) [ 4385.357838] Lustre: DEBUG MARKER: == sanity test 27ca: one stripe on specified OST ========= 04:08:23 (1787386103) [ 4395.376288] Lustre: DEBUG MARKER: == sanity test 27cb: two stripes on specified OSTs ======= 04:08:33 (1787386113) [ 4405.513786] Lustre: DEBUG MARKER: == sanity test 27cc: two stripes on the same OST ========= 04:08:42 (1787386122) [ 4414.867450] Lustre: DEBUG MARKER: == sanity test 27cd: four stripes on two OSTs ============ 04:08:52 (1787386132) [ 4423.854556] Lustre: DEBUG MARKER: == sanity test 27ce: more stripes than OSTs with -o ====== 04:09:01 (1787386141) [ 4432.510926] Lustre: DEBUG MARKER: == sanity test 27cf: 'setstripe -o' on inactive OSTs should return error ========================================================== 04:09:10 (1787386150) [ 4433.626264] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4436.636505] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4436.650085] Lustre: Skipped 8 previous similar messages [ 4436.672537] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4436.682491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.711843] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4436.713098] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.740381] Lustre: Skipped 6 previous similar messages [ 4446.758039] Lustre: DEBUG MARKER: == sanity test 27cg: 1000 shouldn't cause too many credits ========================================================== 04:09:24 (1787386164) [ 4454.337134] Lustre: 13553:0:(osd_handler.c:2191:osd_trans_start()) lustre-MDT0000: credits 30549 > trans_max 3200 [ 4454.351473] Lustre: 13553:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 4/16/0, destroy: 1/4/0 [ 4454.360367] Lustre: 13553:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 2003/2003/0, xattr_set: 3004/28292/0 [ 4454.375431] Lustre: 13553:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 20/142/0, punch: 0/0/0, quota 0/0/0 [ 4454.384141] Lustre: 13553:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 5/84/0, delete: 2/5/0 [ 4454.394285] Lustre: 13553:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 2/2/0 [ 4454.407116] CPU: 2 PID: 13553 Comm: mdt00_004 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 [ 4454.416057] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4454.423050] Call Trace: [ 4454.427086] ? dump_stack+0xbb/0x10e [ 4454.439913] ? osd_trans_start+0x8db/0x9d0 [osd_ldiskfs] [ 4454.444959] ? dt_trans_start+0x1c/0x70 [obdclass] [ 4454.450200] ? top_trans_start+0x4fd/0xd00 [ptlrpc] [ 4454.454712] ? lod_declare_ref_add+0x1a/0x30 [lod] [ 4454.461286] ? dt_declare_ref_add+0x6c/0x200 [obdclass] [ 4454.480098] ? lod_trans_start+0x109/0x4c0 [lod] [ 4454.483345] ? mdd_declare_links_add+0x60/0x90 [mdd] [ 4454.494207] ? mdd_trans_start+0x18/0x30 [mdd] [ 4454.496380] ? mdd_unlink+0x7a4/0x13c0 [mdd] [ 4454.514412] ? mdt_reint_unlink+0x14aa/0x1a30 [mdt] [ 4454.521447] ? mdt_reint_rec+0x139/0x2b0 [mdt] [ 4454.532896] ? mdt_reint_internal+0x693/0xdc0 [mdt] [ 4454.539388] ? mdt_reint+0x163/0x190 [mdt] [ 4454.546221] ? tgt_handle_request0+0x137/0xaf0 [ptlrpc] [ 4454.552938] ? tgt_request_handle+0x575/0x1f70 [ptlrpc] [ 4454.559297] ? ptlrpc_server_handle_request+0x443/0x13b0 [ptlrpc] [ 4454.566335] ? lprocfs_counter_add+0x15b/0x210 [obdclass] [ 4454.580412] ? ptlrpc_main+0xce8/0x1400 [ptlrpc] [ 4454.588110] ? ptlrpc_wait_event+0x690/0x690 [ptlrpc] [ 4454.601315] ? kthread+0x1d1/0x200 [ 4454.608894] ? set_kthread_struct+0x70/0x70 [ 4454.615768] ? ret_from_fork+0x1f/0x30 [ 4471.492123] Lustre: DEBUG MARKER: == sanity test 27d: create file with default settings ==== 04:09:48 (1787386188) [ 4482.265112] Lustre: DEBUG MARKER: == sanity test 27e: setstripe existing file (should return error) ========================================================== 04:09:59 (1787386199) [ 4490.071963] Lustre: DEBUG MARKER: == sanity test 27f: setstripe with bad stripe size (should return error) ========================================================== 04:10:07 (1787386207) [ 4499.966183] Lustre: DEBUG MARKER: == sanity test 27g: /home/green/git/lustre-release/lustre/utils/lfs getstripe with no objects ========================================================== 04:10:17 (1787386217) [ 4509.912678] Lustre: DEBUG MARKER: == sanity test 27ga: /home/green/git/lustre-release/lustre/utils/lfs getstripe with missing file (should return error) ========================================================== 04:10:27 (1787386227) [ 4518.666778] Lustre: DEBUG MARKER: == sanity test 27i: /home/green/git/lustre-release/lustre/utils/lfs getstripe with some objects ========================================================== 04:10:36 (1787386236) [ 4527.092707] Lustre: DEBUG MARKER: == sanity test 27j: setstripe with bad stripe offset (should return error) ========================================================== 04:10:44 (1787386244) [ 4534.757803] Lustre: DEBUG MARKER: == sanity test 27k: limit i_blksize for broken user apps ========================================================== 04:10:52 (1787386252) [ 4542.699641] Lustre: DEBUG MARKER: == sanity test 27l: check setstripe permissions (should return error) ========================================================== 04:11:00 (1787386260) [ 4550.494575] Lustre: DEBUG MARKER: SKIP: sanity test_27m skipping SLOW test 27m [ 4552.568432] Lustre: DEBUG MARKER: == sanity test 27n: create file with some full OSTs ====== 04:11:10 (1787386270) [ 4569.631512] Lustre: *** cfs_fail_loc=215, val=0*** [ 4569.983746] Lustre: *** cfs_fail_loc=215, val=0*** [ 4575.200869] Lustre: *** cfs_fail_loc=215, val=0*** [ 4575.224260] Lustre: Skipped 1 previous similar message [ 4601.084832] Lustre: DEBUG MARKER: == sanity test 27o: create file with all full OSTs (should error) ========================================================== 04:11:58 (1787386318) [ 4616.160549] Lustre: *** cfs_fail_loc=215, val=0*** [ 4616.171865] Lustre: Skipped 1 previous similar message [ 4621.280371] Lustre: *** cfs_fail_loc=215, val=0*** [ 4621.284569] Lustre: Skipped 3 previous similar messages [ 4623.904995] LustreError: 36894:0:(lod_qos.c:1405:lod_ost_alloc_specific()) can't lstripe objid [0x200000402:0xd50f:0x0]: have 0 want 1 [ 4626.401928] Lustre: *** cfs_fail_loc=215, val=0*** [ 4626.409962] Lustre: Skipped 2 previous similar messages [ 4636.645219] Lustre: *** cfs_fail_loc=215, val=0*** [ 4636.648471] Lustre: Skipped 7 previous similar messages [ 4655.202202] Lustre: DEBUG MARKER: == sanity test 27oo: don't let few threads to reserve too many objects ========================================================== 04:12:52 (1787386372) [ 4683.681216] Lustre: Failing over lustre-OST0000 [ 4683.841666] Lustre: server umount lustre-OST0000 complete [ 4687.843531] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4687.849283] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4687.856768] LustreError: Skipped 1 previous similar message [ 4687.868078] Lustre: Skipped 1 previous similar message [ 4687.870206] LustreError: 47664:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4687.917103] LustreError: 47664:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 56 previous similar messages [ 4701.863583] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4702.203698] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4702.234044] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4702.245640] Lustre: Skipped 1 previous similar message [ 4703.681666] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4703.698518] Lustre: Skipped 1 previous similar message [ 4704.251200] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4704.251363] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4704.260674] Lustre: Skipped 1 previous similar message [ 4704.271153] Lustre: Skipped 1 previous similar message [ 4709.417499] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 4716.482458] Lustre: DEBUG MARKER: == sanity test 27p: append to a truncated file with some full OSTs ========================================================== 04:13:54 (1787386434) [ 4740.065914] Lustre: *** cfs_fail_loc=215, val=0*** [ 4740.069317] Lustre: Skipped 3 previous similar messages [ 4768.199720] Lustre: DEBUG MARKER: == sanity test 27q: append to truncated file with all OSTs full (should error) ========================================================== 04:14:45 (1787386485) [ 4780.514652] Lustre: *** cfs_fail_loc=215, val=1*** [ 4780.523387] Lustre: Skipped 1 previous similar message [ 4781.437847] LustreError: 36894:0:(lod_qos.c:1405:lod_ost_alloc_specific()) can't lstripe objid [0x200000402:0xd526:0x0]: have 0 want 1 [ 4788.367507] LustreError: 6509:0:(lod_qos.c:1405:lod_ost_alloc_specific()) can't lstripe objid [0x200000402:0xd52a:0x0]: have 0 want 1 [ 4815.821263] Lustre: DEBUG MARKER: == sanity test 27r: stripe file with some full OSTs (shouldn't LBUG) =========================================================== 04:15:33 (1787386533) [ 4857.563776] Lustre: DEBUG MARKER: == sanity test 27s: lsm_xfersize overflow (should error) (bug 10725) ========================================================== 04:16:15 (1787386575) [ 4864.241831] Lustre: DEBUG MARKER: == sanity test 27t: check that utils parse path correctly ========================================================== 04:16:22 (1787386582) [ 4871.427975] Lustre: DEBUG MARKER: == sanity test 27u: skip object creation on OSC w/o objects ========================================================== 04:16:29 (1787386589) [ 4875.569696] Lustre: *** cfs_fail_loc=139, val=0*** [ 4876.070098] Lustre: *** cfs_fail_loc=139, val=0*** [ 4876.071875] Lustre: Skipped 63 previous similar messages [ 4877.074970] Lustre: *** cfs_fail_loc=139, val=0*** [ 4877.077504] Lustre: Skipped 165 previous similar messages [ 4879.087802] Lustre: *** cfs_fail_loc=139, val=0*** [ 4879.089899] Lustre: Skipped 363 previous similar messages [ 4883.100633] Lustre: *** cfs_fail_loc=139, val=0*** [ 4883.110141] Lustre: Skipped 627 previous similar messages [ 4930.691255] Lustre: DEBUG MARKER: == sanity test 27v: skip object creation on slow OST ===== 04:17:27 (1787386647) [ 4934.113194] Lustre: *** cfs_fail_loc=215, val=1*** [ 4934.127651] Lustre: Skipped 22 previous similar messages [ 4936.050799] LustreError: 36894:0:(lod_qos.c:1405:lod_ost_alloc_specific()) can't lstripe objid [0x200000402:0xd73c:0x0]: have 0 want 1 [ 4942.494902] LustreError: 6510:0:(lod_qos.c:1405:lod_ost_alloc_specific()) can't lstripe objid [0x200000402:0xd740:0x0]: have 0 want 1 [ 4990.360417] Lustre: DEBUG MARKER: == sanity test 27w: check /home/green/git/lustre-release/lustre/utils/lfs setstripe -S and getstrip -d options ========================================================== 04:18:28 (1787386708) [ 4997.327436] Lustre: DEBUG MARKER: == sanity test 27wa: check /home/green/git/lustre-release/lustre/utils/lfs setstripe -c -i options ========================================================== 04:18:35 (1787386715) [ 5004.558229] Lustre: DEBUG MARKER: == sanity test 27x: create files while OST0 is degraded == 04:18:42 (1787386722) [ 5026.092564] Lustre: DEBUG MARKER: == sanity test 27y: create files while OST0 is degraded and the rest inactive ========================================================== 04:19:03 (1787386743) [ 5030.715711] Lustre: setting import lustre-OST0001_UUID INACTIVE by administrator request [ 5030.720786] Lustre: Skipped 1 previous similar message [ 5049.531830] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5049.540607] Lustre: Skipped 1 previous similar message [ 5049.544224] Lustre: lustre-OST0001: Client lustre-MDT0001-mdtlov_UUID (at 0@lo) reconnecting [ 5049.547283] Lustre: Skipped 1 previous similar message [ 5049.551963] LustreError: lustre-OST0001-osc-MDT0001: This client was evicted by lustre-OST0001; in progress operations using this service will fail. [ 5049.560637] Lustre: lustre-OST0001-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 5049.567816] Lustre: Skipped 1 previous similar message [ 5050.494192] Lustre: lustre-OST0001: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5050.501676] LustreError: lustre-OST0001-osc-MDT0000: This client was evicted by lustre-OST0001; in progress operations using this service will fail. [ 5070.033364] Lustre: DEBUG MARKER: == sanity test 27z: check SEQ/OID on the MDT and OST filesystems ========================================================== 04:19:48 (1787386788) [ 5078.861946] Lustre: DEBUG MARKER: check file /mnt/lustre/d27z.sanity/f27z.sanity-1 [ 5080.921877] Lustre: DEBUG MARKER: FID seq 0x200000402, oid 0xd769 ver 0x0 [ 5083.091295] Lustre: DEBUG MARKER: LOV seq 0x200000402, oid 0xd769, count: 1 [ 5084.733992] Lustre: DEBUG MARKER: want: stripe:0 ost:0 oid:867/0x363 seq:0x280000401 [ 5087.961575] Lustre: DEBUG MARKER: check file /mnt/lustre/d27z.sanity/f27z.sanity-2 [ 5090.239873] Lustre: DEBUG MARKER: FID seq 0x200000402, oid 0xd76b ver 0x0 [ 5092.316775] Lustre: DEBUG MARKER: LOV seq 0x200000402, oid 0xd76b, count: 2 [ 5094.336755] Lustre: DEBUG MARKER: want: stripe:0 ost:1 oid:1379/0x563 seq:0x2c0000401 [ 5097.265768] Lustre: DEBUG MARKER: want: stripe:1 ost:0 oid:868/0x364 seq:0x280000401 [ 5105.477332] Lustre: DEBUG MARKER: == sanity test 27A: check filesystem-wide default LOV EA values ========================================================== 04:20:23 (1787386823) [ 5113.886367] Lustre: DEBUG MARKER: == sanity test 27B: call setstripe on open unlinked file/rename victim ========================================================== 04:20:31 (1787386831) [ 5120.990866] Lustre: DEBUG MARKER: == sanity test 27Ca: check full striping across all OSTs ========================================================== 04:20:39 (1787386839) [ 5129.897415] Lustre: DEBUG MARKER: == sanity test 27Cb: more stripes than OSTs with -C ====== 04:20:47 (1787386847) [ 5138.313958] Lustre: DEBUG MARKER: == sanity test 27Cc: fewer stripes than OSTs does not set overstriping ========================================================== 04:20:56 (1787386856) [ 5147.692553] Lustre: DEBUG MARKER: == sanity test 27Cd: test maximum stripe count =========== 04:21:05 (1787386865) [ 5156.312761] Lustre: lustre-OST0001-osc-MDT0001: update sequence from 0x2c0000400 to 0x2c0000402 [ 5156.340606] Lustre: lustre-OST0000-osc-MDT0001: update sequence from 0x280000400 to 0x280000402 [ 5156.632758] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x280000401 to 0x280000403 [ 5156.684140] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x2c0000401 to 0x2c0000403 [ 5180.470217] Lustre: 6510:0:(osd_handler.c:2191:osd_trans_start()) lustre-MDT0000: credits 60549 > trans_max 3200 [ 5180.479516] Lustre: 6510:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 4/16/0, destroy: 1/4/0 [ 5180.485267] Lustre: 6510:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 4003/4003/0, xattr_set: 6004/56292/0 [ 5180.490785] Lustre: 6510:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 20/142/0, punch: 0/0/0, quota 0/0/0 [ 5180.499830] Lustre: 6510:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 5/84/0, delete: 2/5/0 [ 5180.505338] Lustre: 6510:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 2/2/0 [ 5180.510449] CPU: 1 PID: 6510 Comm: mdt00_002 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 [ 5180.519674] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5180.525583] Call Trace: [ 5180.526681] ? dump_stack+0xbb/0x10e [ 5180.529828] ? osd_trans_start+0x8db/0x9d0 [osd_ldiskfs] [ 5180.534337] ? dt_trans_start+0x1c/0x70 [obdclass] [ 5180.536579] ? top_trans_start+0x4fd/0xd00 [ptlrpc] [ 5180.538953] ? lod_declare_ref_add+0x1a/0x30 [lod] [ 5180.541324] ? dt_declare_ref_add+0x6c/0x200 [obdclass] [ 5180.543660] ? lod_trans_start+0x109/0x4c0 [lod] [ 5180.547552] ? mdd_declare_links_add+0x60/0x90 [mdd] [ 5180.549856] ? mdd_trans_start+0x18/0x30 [mdd] [ 5180.552437] ? mdd_unlink+0x7a4/0x13c0 [mdd] [ 5180.553809] ? mdt_reint_unlink+0x14aa/0x1a30 [mdt] [ 5180.555715] ? mdt_reint_rec+0x139/0x2b0 [mdt] [ 5180.557653] ? mdt_reint_internal+0x693/0xdc0 [mdt] [ 5180.559488] ? mdt_reint+0x163/0x190 [mdt] [ 5180.560890] ? tgt_handle_request0+0x137/0xaf0 [ptlrpc] [ 5180.564804] ? tgt_request_handle+0x575/0x1f70 [ptlrpc] [ 5180.568340] ? ptlrpc_server_handle_request+0x443/0x13b0 [ptlrpc] [ 5180.571811] ? lprocfs_counter_add+0x15b/0x210 [obdclass] [ 5180.575600] ? ptlrpc_main+0xce8/0x1400 [ptlrpc] [ 5180.579663] ? ptlrpc_wait_event+0x690/0x690 [ptlrpc] [ 5180.582943] ? kthread+0x1d1/0x200 [ 5180.584500] ? set_kthread_struct+0x70/0x70 [ 5180.586487] ? ret_from_fork+0x1f/0x30 [ 5180.591016] Lustre: 6510:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-MDT0000: opcode 0: before 3202 < left 4003, rollback = 0 [ 5197.056801] Lustre: DEBUG MARKER: == sanity test 27Ce: test pool with overstriping ========= 04:21:54 (1787386914) [ 5225.258292] Lustre: DEBUG MARKER: == sanity test 27Cf: test default inheritance with overstriping ========================================================== 04:22:22 (1787386942) [ 5232.758436] Lustre: DEBUG MARKER: == sanity test 27Cg: test setstripe with wrong OST idx === 04:22:30 (1787386950) [ 5240.595937] Lustre: DEBUG MARKER: == sanity test 27Ci: add an overstriping component ======= 04:22:38 (1787386958) [ 5248.522978] Lustre: 36894:0:(osd_handler.c:2191:osd_trans_start()) lustre-MDT0000: credits 3579 > trans_max 3200 [ 5248.529315] Lustre: 36894:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 4/16/0, destroy: 1/4/0 [ 5248.534352] Lustre: 36894:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 3 previous similar messages [ 5248.540818] Lustre: 36894:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 205/205/0, xattr_set: 307/3120/0 [ 5248.546602] Lustre: 36894:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 3 previous similar messages [ 5248.551710] Lustre: 36894:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 20/142/0, punch: 0/0/0, quota 0/0/0 [ 5248.556956] Lustre: 36894:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 3 previous similar messages [ 5248.561146] Lustre: 36894:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 5/84/0, delete: 2/5/0 [ 5248.565200] Lustre: 36894:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 3 previous similar messages [ 5248.569916] Lustre: 36894:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 2/2/0 [ 5248.573863] Lustre: 36894:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 3 previous similar messages [ 5248.578086] CPU: 3 PID: 36894 Comm: mdt00_005 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 [ 5248.582496] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 5248.586265] Call Trace: [ 5248.587299] ? dump_stack+0xbb/0x10e [ 5248.589274] ? osd_trans_start+0x8db/0x9d0 [osd_ldiskfs] [ 5248.591145] ? dt_trans_start+0x1c/0x70 [obdclass] [ 5248.596295] ? top_trans_start+0x4fd/0xd00 [ptlrpc] [ 5248.598980] ? lod_declare_ref_add+0x1a/0x30 [lod] [ 5248.601286] ? dt_declare_ref_add+0x6c/0x200 [obdclass] [ 5248.604163] ? lod_trans_start+0x109/0x4c0 [lod] [ 5248.607102] ? mdd_declare_links_add+0x60/0x90 [mdd] [ 5248.609590] ? mdd_trans_start+0x18/0x30 [mdd] [ 5248.611715] ? mdd_unlink+0x7a4/0x13c0 [mdd] [ 5248.613774] ? mdt_reint_unlink+0x14aa/0x1a30 [mdt] [ 5248.615820] ? mdt_reint_rec+0x139/0x2b0 [mdt] [ 5248.618339] ? mdt_reint_internal+0x693/0xdc0 [mdt] [ 5248.620594] ? mdt_reint+0x163/0x190 [mdt] [ 5248.622849] ? tgt_handle_request0+0x137/0xaf0 [ptlrpc] [ 5248.626065] ? tgt_request_handle+0x575/0x1f70 [ptlrpc] [ 5248.628721] ? ptlrpc_server_handle_request+0x443/0x13b0 [ptlrpc] [ 5248.631792] ? lprocfs_counter_add+0x15b/0x210 [obdclass] [ 5248.634135] ? ptlrpc_main+0xce8/0x1400 [ptlrpc] [ 5248.636782] ? ptlrpc_wait_event+0x690/0x690 [ptlrpc] [ 5248.641946] ? kthread+0x1d1/0x200 [ 5248.644283] ? set_kthread_struct+0x70/0x70 [ 5248.646751] ? ret_from_fork+0x1f/0x30 [ 5250.991701] Lustre: DEBUG MARKER: == sanity test 27Cj: overstriping with -C for max values in multiple of targets ========================================================== 04:22:48 (1787386968) [ 5258.585475] Lustre: DEBUG MARKER: == sanity test 27D: validate llapi_layout API ============ 04:22:56 (1787386976) [ 5291.160343] Lustre: DEBUG MARKER: == sanity test 27E: check that default extended attribute size properly increases ========================================================== 04:23:29 (1787387009) [ 5297.976787] Lustre: DEBUG MARKER: == sanity test 27F: Client resend delayed layout creation with non-zero size ========================================================== 04:23:36 (1787387016) [ 5300.702638] Lustre: Failing over lustre-OST0000 [ 5300.850028] Lustre: server umount lustre-OST0000 complete [ 5302.849647] LustreError: 47784:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 192.168.202.49@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5302.872595] LustreError: 47784:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 5303.274761] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5303.286290] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5303.303218] Lustre: Skipped 2 previous similar messages [ 5304.000098] Lustre: Failing over lustre-OST0001 [ 5304.121721] Lustre: server umount lustre-OST0001 complete [ 5312.501668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5312.792397] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 5312.812421] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 5318.862775] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 5319.150233] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 5320.160291] Lustre: 3638:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787387023/real 1787387023] req@ffff9da3434e2680 x1874205699369088/t0(0) o400->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787387039 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 5321.190057] Lustre: 3635:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787387024/real 1787387024] req@ffff9da472a92a00 x1874205699369344/t0(0) o13->lustre-OST0001-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1787387040 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'osp-pre-1-1.0' uid:0 gid:0 projid:4294967295 [ 5321.203736] Lustre: 3635:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 5322.210099] Lustre: 3636:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787387025/real 1787387025] req@ffff9da4669fdf80 x1874205699369856/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 1 dl 1787387041 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 [ 5323.521863] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5323.523321] Lustre: lustre-OST0000: Recovery over after 0:04, of 3 clients 3 recovered and 0 were evicted. [ 5323.529348] Lustre: Skipped 1 previous similar message [ 5325.279418] Lustre: 3636:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787387028/real 1787387028] req@ffff9da473d6f100 x1874205699370880/t0(0) o400->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787387044 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 5325.302907] Lustre: 3636:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 5327.623338] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5327.940946] Lustre: lustre-OST0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 5329.113049] Lustre: lustre-OST0001: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 5334.819365] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug all all [ 5344.405876] Lustre: DEBUG MARKER: == sanity test 27G: Clear OST pool from stripe =========== 04:24:22 (1787387062) [ 5370.643913] Lustre: DEBUG MARKER: == sanity test 27H: Set specific OSTs stripe ============= 04:24:48 (1787387088) [ 5372.048589] Lustre: DEBUG MARKER: SKIP: sanity test_27H needs >= 3 OSTs [ 5373.496342] Lustre: DEBUG MARKER: == sanity test 27I: check that root dir striping does not break parent dir one ========================================================== 04:24:51 (1787387091) [ 5397.317085] Lustre: DEBUG MARKER: == sanity test 27Ia: check that root dir pool is dropped with conflict parent dir settings ========================================================== 04:25:15 (1787387115) [ 5423.624531] Lustre: DEBUG MARKER: == sanity test 27J: basic ops on file with foreign LOV === 04:25:41 (1787387141) [ 5432.512613] Lustre: DEBUG MARKER: == sanity test 27K: basic ops on dir with foreign LMV ==== 04:25:50 (1787387150) [ 5442.122932] Lustre: DEBUG MARKER: == sanity test 27Ke: test enable_foreign_dir and enable_foreign_dir_gid ========================================================== 04:25:59 (1787387159) [ 5474.137243] Lustre: DEBUG MARKER: == sanity test 27L: lfs pool_list gives correct pool name ========================================================== 04:26:31 (1787387191) [ 5495.819806] Lustre: DEBUG MARKER: == sanity test 27M: test O_APPEND striping =============== 04:26:53 (1787387213) [ 5558.991327] Lustre: DEBUG MARKER: == sanity test 27N: lctl pool_list on separate MGS gives correct pool name ========================================================== 04:27:56 (1787387276) [ 5560.820718] Lustre: DEBUG MARKER: SKIP: sanity test_27N needs separate MGS/MDT [ 5563.259713] Lustre: DEBUG MARKER: == sanity test 27O: basic ops on foreign file of symlink type ========================================================== 04:28:00 (1787387280) [ 5574.056385] Lustre: DEBUG MARKER: == sanity test 27P: basic ops on foreign dir of foreign_symlink type ========================================================== 04:28:12 (1787387292) [ 5585.076734] Lustre: DEBUG MARKER: == sanity test 27Q: llapi_file_get_stripe() works on symlinks ========================================================== 04:28:22 (1787387302) [ 5593.551497] Lustre: DEBUG MARKER: == sanity test 27R: test max_stripecount limitation when stripe count is set to -1 ========================================================== 04:28:31 (1787387311) [ 5605.445520] Lustre: DEBUG MARKER: == sanity test 27T: no eio on close on partial write due to enosp ========================================================== 04:28:43 (1787387323) [ 5616.367586] Lustre: *** cfs_fail_loc=255, val=0*** [ 5626.198832] Lustre: DEBUG MARKER: == sanity test 27U: append pool and stripe count work with composite default layout ========================================================== 04:29:03 (1787387343) [ 5685.459702] Lustre: DEBUG MARKER: == sanity test 27V: creating widely striped file races with deactivating OST ========================================================== 04:30:03 (1787387403) [ 5687.599393] Lustre: DEBUG MARKER: SKIP: sanity test_27V needs >= 4 OSTs [ 5690.314120] Lustre: DEBUG MARKER: == sanity test 27W: test enable_setstripe_gid ============ 04:30:07 (1787387407) [ 5698.991697] Lustre: DEBUG MARKER: == sanity test 27X: lfs migrate honors --overstripe-count option ========================================================== 04:30:16 (1787387416) [ 5707.116853] Lustre: DEBUG MARKER: == sanity test 28: create/mknod/mkdir with bad file types ====================================================================== 04:30:25 (1787387425) [ 5716.615541] Lustre: DEBUG MARKER: == sanity test 29: IT_GETATTR regression ====================================================================================== 04:30:34 (1787387434) [ 5720.114665] Lustre: DEBUG MARKER: first d29 [ 5721.927224] Lustre: DEBUG MARKER: second d29 [ 5723.556637] Lustre: DEBUG MARKER: done [ 5730.634675] Lustre: DEBUG MARKER: == sanity test 30a: execute binary from Lustre (execve) ======================================================================== 04:30:48 (1787387448) [ 5737.771754] Lustre: DEBUG MARKER: == sanity test 30b: execute binary from Lustre as non-root ===================================================================== 04:30:55 (1787387455) [ 5745.888486] Lustre: DEBUG MARKER: == sanity test 30c: execute binary from Lustre without read perms ============================================================== 04:31:03 (1787387463) [ 5755.194611] Lustre: DEBUG MARKER: == sanity test 30d: execute binary from Lustre while clear locks ========================================================== 04:31:13 (1787387473) [ 5995.385299] Lustre: DEBUG MARKER: == sanity test 31a: open-unlink file ============================================================================================ 04:35:12 (1787387712) [ 6003.633270] Lustre: DEBUG MARKER: == sanity test 31b: unlink file with multiple links while open ================================================================= 04:35:21 (1787387721) [ 6011.600428] Lustre: DEBUG MARKER: == sanity test 31c: open-unlink file with multiple links ======================================================================= 04:35:29 (1787387729) [ 6020.759578] Lustre: DEBUG MARKER: == sanity test 31d: remove of open directory =================================================================================== 04:35:38 (1787387738) [ 6028.691937] Lustre: DEBUG MARKER: == sanity test 31e: remove of open non-empty directory ========================================================================= 04:35:46 (1787387746) [ 6037.011732] Lustre: DEBUG MARKER: == sanity test 31f: remove of open directory with open-unlink file ============================================================= 04:35:54 (1787387754) [ 6051.303597] Lustre: DEBUG MARKER: == sanity test 31g: cross directory link================== 04:36:09 (1787387769) [ 6059.402777] Lustre: DEBUG MARKER: == sanity test 31h: cross directory link under child========================================================================= 04:36:17 (1787387777) [ 6067.600626] Lustre: DEBUG MARKER: == sanity test 31i: cross directory link under parent========================================================================= 04:36:25 (1787387785) [ 6075.497123] Lustre: DEBUG MARKER: == sanity test 31j: link for directory =================== 04:36:33 (1787387793) [ 6083.557560] Lustre: DEBUG MARKER: == sanity test 31k: link to file: the same, non-existing, dir ========================================================== 04:36:41 (1787387801) [ 6091.201641] Lustre: DEBUG MARKER: == sanity test 31l: link to file: target dir has trailing slash ========================================================== 04:36:49 (1787387809) [ 6098.148353] Lustre: DEBUG MARKER: == sanity test 31m: link to file: the same, non-existing, dir ========================================================== 04:36:56 (1787387816) [ 6106.006223] Lustre: DEBUG MARKER: == sanity test 31n: check link count of unlinked file ==== 04:37:03 (1787387823) [ 6113.967219] Lustre: DEBUG MARKER: == sanity test 31o: duplicate hard links with same filename ========================================================== 04:37:11 (1787387831) [ 6209.347486] Lustre: DEBUG MARKER: == sanity test 31p: remove of open striped directory ===== 04:38:47 (1787387927) [ 6217.770849] Lustre: DEBUG MARKER: == sanity test 31q: create striped directory on specific MDTs ========================================================== 04:38:55 (1787387935) [ 6219.559963] Lustre: DEBUG MARKER: SKIP: sanity test_31q needs >= 3 MDTs [ 6222.097673] Lustre: DEBUG MARKER: == sanity test 31r: open-rename(replace) race ============ 04:38:59 (1787387939) [ 6231.071854] Lustre: DEBUG MARKER: == sanity test 32a: stat d32a/ext2-mountpoint/.. =============================================================================== 04:39:09 (1787387949) [ 6238.618980] Lustre: DEBUG MARKER: == sanity test 32b: open d32b/ext2-mountpoint/.. =============================================================================== 04:39:16 (1787387956) [ 6246.867933] Lustre: DEBUG MARKER: == sanity test 32c: stat d32c/ext2-mountpoint/../d2/test_dir =================================================================== 04:39:24 (1787387964) [ 6255.674597] Lustre: DEBUG MARKER: == sanity test 32d: open d32d/ext2-mountpoint/../d2/test_dir ========================================================== 04:39:33 (1787387973) [ 6264.872264] Lustre: DEBUG MARKER: == sanity test 32e: stat d32e/symlink->tmp/symlink->lustre-subdir ========================================================== 04:39:42 (1787387982) [ 6273.613402] Lustre: DEBUG MARKER: == sanity test 32f: open d32f/symlink->tmp/symlink->lustre-subdir ========================================================== 04:39:51 (1787387991) [ 6282.703753] Lustre: DEBUG MARKER: == sanity test 32g: stat d32g/symlink->tmp/symlink->lustre-subdir/2 ========================================================== 04:40:00 (1787388000) [ 6291.816961] Lustre: DEBUG MARKER: == sanity test 32h: open d32h/symlink->tmp/symlink->lustre-subdir/2 ========================================================== 04:40:09 (1787388009) [ 6300.132651] Lustre: DEBUG MARKER: == sanity test 32i: stat d32i/ext2-mountpoint/../test_file ===================================================================== 04:40:18 (1787388018) [ 6308.504874] Lustre: DEBUG MARKER: == sanity test 32j: open d32j/ext2-mountpoint/../test_file ===================================================================== 04:40:26 (1787388026) [ 6318.302742] Lustre: DEBUG MARKER: == sanity test 32k: stat d32k/ext2-mountpoint/../d2/test_file ================================================================== 04:40:35 (1787388035) [ 6329.181378] Lustre: DEBUG MARKER: == sanity test 32l: open d32l/ext2-mountpoint/../d2/test_file ================================================================== 04:40:46 (1787388046) [ 6340.348919] Lustre: DEBUG MARKER: == sanity test 32m: stat d32m/symlink->tmp/symlink->lustre-root ================================================================ 04:40:57 (1787388057) [ 6348.741810] Lustre: DEBUG MARKER: == sanity test 32n: open d32n/symlink->tmp/symlink->lustre-root ================================================================ 04:41:06 (1787388066) [ 6357.706041] Lustre: DEBUG MARKER: == sanity test 32o: stat d32o/symlink->tmp/symlink->lustre-root/ ========================================================== 04:41:15 (1787388075) [ 6365.994835] Lustre: DEBUG MARKER: == sanity test 32p: open d32p/symlink->tmp/symlink->lustre-root/ ========================================================== 04:41:23 (1787388083) [ 6368.106988] Lustre: DEBUG MARKER: 32p_1 [ 6369.923903] Lustre: DEBUG MARKER: 32p_2 [ 6371.500911] Lustre: DEBUG MARKER: 32p_3 [ 6373.968955] Lustre: DEBUG MARKER: 32p_4 [ 6376.397495] Lustre: DEBUG MARKER: 32p_5 [ 6378.572687] Lustre: DEBUG MARKER: 32p_6 [ 6380.832800] Lustre: DEBUG MARKER: 32p_7 [ 6382.640457] Lustre: DEBUG MARKER: 32p_8 [ 6385.913893] Lustre: DEBUG MARKER: 32p_9 [ 6387.898775] Lustre: DEBUG MARKER: 32p_10 [ 6394.979509] Lustre: DEBUG MARKER: == sanity test 32q: stat follows mountpoints in Lustre (should return error) ========================================================== 04:41:53 (1787388113) [ 6404.263272] Lustre: DEBUG MARKER: == sanity test 32r: opendir follows mountpoints in Lustre (should return error) ========================================================== 04:42:01 (1787388121) [ 6413.575210] Lustre: DEBUG MARKER: == sanity test 33aa: write file with mode 444 (should return error) ========================================================== 04:42:11 (1787388131) [ 6416.061390] Lustre: DEBUG MARKER: 33_1 [ 6418.487355] Lustre: DEBUG MARKER: 33_2 [ 6426.175091] Lustre: DEBUG MARKER: == sanity test 33a: test open file(mode=0444) with O_RDWR (should return error) ========================================================== 04:42:24 (1787388144) [ 6434.632459] Lustre: DEBUG MARKER: == sanity test 33b: test open file with malformed flags (No panic) ========================================================== 04:42:32 (1787388152) [ 6442.664273] Lustre: DEBUG MARKER: == sanity test 33c: test write_bytes stats =============== 04:42:40 (1787388160) [ 6451.902938] Lustre: DEBUG MARKER: == sanity test 33d: openfile with 444 modes and malformed flags under remote dir ========================================================== 04:42:49 (1787388169) [ 6460.759062] Lustre: DEBUG MARKER: == sanity test 33e: mkdir and striped directory should have same mode ========================================================== 04:42:58 (1787388178) [ 6471.410110] Lustre: DEBUG MARKER: == sanity test 33f: nonroot user can create, access, and remove a striped directory ========================================================== 04:43:09 (1787388189) [ 6483.357378] Lustre: DEBUG MARKER: == sanity test 33g: nonroot user create already existing root created file ========================================================== 04:43:21 (1787388201) [ 6491.058881] Lustre: DEBUG MARKER: == sanity test 33h: temp file is located on the same MDT as target (crush) ========================================================== 04:43:28 (1787388208) [ 6652.343985] Lustre: DEBUG MARKER: == sanity test 33hh: temp file is located on the same MDT as target (crush2) ========================================================== 04:46:09 (1787388369) [ 6817.867293] Lustre: DEBUG MARKER: == sanity test 33i: striped directory can be accessed when one MDT is down ========================================================== 04:48:55 (1787388535) [ 6874.083272] Lustre: lustre-MDT0001: Client 1d290810-da2d-4434-ac61-e9abbf93f958 (at 192.168.202.49@tcp) reconnecting [ 6876.496562] Lustre: DEBUG MARKER: == sanity test 33j: lfs setdirstripe -D -i x,y,x should fail ========================================================== 04:49:53 (1787388593) [ 6884.572222] Lustre: DEBUG MARKER: == sanity test 34a: truncate file that has not been opened ===================================================================== 04:50:02 (1787388602) [ 6893.638964] Lustre: DEBUG MARKER: == sanity test 34b: O_RDONLY opening file doesn't create objects =============================================================== 04:50:11 (1787388611) [ 6901.671811] Lustre: DEBUG MARKER: == sanity test 34c: O_RDWR opening file-with-size works ======================================================================== 04:50:19 (1787388619) [ 6910.489320] Lustre: DEBUG MARKER: == sanity test 34d: write to sparse file ======================================================================================= 04:50:27 (1787388627) [ 6920.498665] Lustre: DEBUG MARKER: == sanity test 34e: create objects, some with size and some without ============================================================ 04:50:37 (1787388637) [ 6931.047168] Lustre: DEBUG MARKER: == sanity test 34f: read from a file with no objects until EOF ================================================================= 04:50:47 (1787388647) [ 6940.671560] Lustre: DEBUG MARKER: == sanity test 34g: truncate long file ========================================================================================= 04:50:58 (1787388658) [ 6950.839467] Lustre: DEBUG MARKER: == sanity test 34h: ftruncate file under grouplock should not block ========================================================== 04:51:08 (1787388668) [ 6964.879437] Lustre: DEBUG MARKER: == sanity test 35a: exec file with mode 444 (should return and not leak) ========================================================== 04:51:22 (1787388682) [ 6973.820251] Lustre: DEBUG MARKER: == sanity test 36a: MDS utime check (mknod, utime) ======= 04:51:31 (1787388691) [ 6984.491393] Lustre: DEBUG MARKER: == sanity test 36b: OST utime check (open, utime) ======== 04:51:41 (1787388701) [ 6995.452990] Lustre: DEBUG MARKER: == sanity test 36c: non-root MDS utime check (mknod, utime) ========================================================== 04:51:52 (1787388712) [ 7004.602914] Lustre: DEBUG MARKER: == sanity test 36d: non-root OST utime check (open, utime) ========================================================== 04:52:01 (1787388721) [ 7013.665854] Lustre: DEBUG MARKER: == sanity test 36e: utime on non-owned file (should return error) ========================================================== 04:52:11 (1787388731) [ 7023.405550] Lustre: DEBUG MARKER: == sanity test 36f: utime on file racing with OST BRW write ==================================================================== 04:52:20 (1787388740) [ 7033.634446] Lustre: DEBUG MARKER: == sanity test 36g: FMD cache expiry =============================================================================== 04:52:31 (1787388751) [ 7088.745703] Lustre: DEBUG MARKER: == sanity test 36h: utime on file racing with OST BRW write ==================================================================== 04:53:26 (1787388806) [ 7097.669646] Lustre: DEBUG MARKER: == sanity test 36i: change mtime on striped directory ==== 04:53:35 (1787388815) [ 7106.556559] Lustre: DEBUG MARKER: == sanity test 38: open a regular file with O_DIRECTORY should return -ENOTDIR ============================================================= 04:53:43 (1787388823) [ 7115.254477] Lustre: DEBUG MARKER: == sanity test 39a: mtime changed on create ============== 04:53:52 (1787388832) [ 7124.983798] Lustre: DEBUG MARKER: == sanity test 39b: mtime change on open, link, unlink, rename ========================================================== 04:54:03 (1787388843) [ 7134.357388] Lustre: DEBUG MARKER: == sanity test 39c: mtime change on rename =============== 04:54:12 (1787388852) [ 7145.552099] Lustre: DEBUG MARKER: == sanity test 39d: create, utime, stat ================== 04:54:23 (1787388863) [ 7153.354697] Lustre: DEBUG MARKER: == sanity test 39e: create, stat, utime, stat ============ 04:54:31 (1787388871) [ 7161.012475] Lustre: DEBUG MARKER: == sanity test 39f: create, stat, sleep, utime, stat ===== 04:54:38 (1787388878) [ 7171.134595] Lustre: DEBUG MARKER: == sanity test 39g: write, chmod, stat =================== 04:54:49 (1787388889) [ 7181.599870] Lustre: DEBUG MARKER: == sanity test 39h: write, utime within one second, stat ========================================================== 04:54:59 (1787388899) [ 7192.854418] Lustre: DEBUG MARKER: == sanity test 39i: write, rename, stat ================== 04:55:09 (1787388909) [ 7204.145144] Lustre: DEBUG MARKER: == sanity test 39j: write, rename, close, stat =========== 04:55:21 (1787388921) [ 7242.785539] Lustre: DEBUG MARKER: == sanity test 39k: write, utime, close, stat ============ 04:56:00 (1787388960) [ 7253.083637] Lustre: DEBUG MARKER: == sanity test 39l: directory atime update =============== 04:56:10 (1787388970) [ 7274.517745] Lustre: DEBUG MARKER: == sanity test 39m: test atime and mtime before 1970 ===== 04:56:32 (1787388992) [ 7285.655349] Lustre: DEBUG MARKER: == sanity test 39n: check that O_NOATIME is honored ====== 04:56:43 (1787389003) [ 7309.141884] Lustre: DEBUG MARKER: == sanity test 39o: directory cached attributes updated after create ========================================================== 04:57:07 (1787389027) [ 7319.668300] Lustre: DEBUG MARKER: == sanity test 39p: remote directory cached attributes updated after create ================================================================== 04:57:17 (1787389037) [ 7330.600539] Lustre: DEBUG MARKER: == sanity test 39r: lazy atime update on OST ============= 04:57:28 (1787389048) [ 7358.403648] Lustre: DEBUG MARKER: == sanity test 39q: close won't zero out atime =========== 04:57:56 (1787389076) [ 7366.915431] Lustre: DEBUG MARKER: == sanity test 39s: relatime is supported ================ 04:58:04 (1787389084) [ 7398.422720] Lustre: DEBUG MARKER: == sanity test 39u: stat race ============================ 04:58:35 (1787389115) [ 7422.144296] Lustre: DEBUG MARKER: == sanity test 40: failed open(O_TRUNC) doesn't truncate ======================================================================= 04:58:59 (1787389139) [ 7429.408486] Lustre: DEBUG MARKER: == sanity test 41: test small file write + fstat =============================================================================== 04:59:07 (1787389147) [ 7437.835635] Lustre: DEBUG MARKER: SKIP: sanity test_42a skipping ALWAYS excluded test 42a [ 7439.549412] Lustre: DEBUG MARKER: SKIP: sanity test_42b skipping ALWAYS excluded test 42b [ 7441.576671] Lustre: DEBUG MARKER: SKIP: sanity test_42c skipping ALWAYS excluded test 42c [ 7443.797893] Lustre: DEBUG MARKER: == sanity test 42d: test complete truncate of file with cached dirty data ========================================================== 04:59:21 (1787389161) [ 7455.618851] Lustre: DEBUG MARKER: == sanity test 42e: verify sub-RPC writes are not done synchronously ========================================================== 04:59:33 (1787389173) [ 7757.068955] Lustre: DEBUG MARKER: == sanity test 43A: execution of file opened for write should return -ETXTBSY ========================================================== 05:04:34 (1787389474) [ 7766.828812] Lustre: DEBUG MARKER: == sanity test 43a: open(RDWR) of file being executed should return -ETXTBSY ========================================================== 05:04:44 (1787389484) [ 7777.022437] Lustre: DEBUG MARKER: == sanity test 43b: truncate of file being executed should return -ETXTBSY ========================================================== 05:04:55 (1787389495) [ 7787.660153] Lustre: DEBUG MARKER: == sanity test 43c: md5sum of copy into lustre =========== 05:05:05 (1787389505) [ 7796.641389] Lustre: DEBUG MARKER: == sanity test 44A: zero length read from a sparse stripe ========================================================== 05:05:14 (1787389514) [ 7804.990448] Lustre: DEBUG MARKER: == sanity test 44a: test sparse pwrite ========================================================================================= 05:05:22 (1787389522) [ 7819.874952] Lustre: DEBUG MARKER: == sanity test 44b: write one byte at offset 0xfffffffe000 ========================================================== 05:05:37 (1787389537) [ 7829.788619] Lustre: DEBUG MARKER: == sanity test 44c: write 1 byte at max_object_bytes - 1 offset ========================================================== 05:05:46 (1787389546) [ 7839.613317] Lustre: DEBUG MARKER: == sanity test 44d: if write at position fails (EFBIG), so should do append ========================================================== 05:05:57 (1787389557) [ 7850.812079] Lustre: DEBUG MARKER: == sanity test 44e: write and read maximal stripes ======= 05:06:08 (1787389568) [ 7887.275460] Lustre: DEBUG MARKER: == sanity test 44f: Check fiemap for sparse files ======== 05:06:44 (1787389604) [ 8813.372457] Lustre: DEBUG MARKER: == sanity test 44g: test overflow in lov_stripe_size ===== 05:22:11 (1787390531) [ 8822.203727] Lustre: DEBUG MARKER: == sanity test 45: osc io page accounting ================ 05:22:19 (1787390539) [ 8831.369832] Lustre: DEBUG MARKER: == sanity test 46: dirtying a previously written page ========================================================================== 05:22:28 (1787390548) [ 8838.867458] Lustre: DEBUG MARKER: == sanity test 48a: Access renamed working dir (should return errors)=========================================================== 05:22:36 (1787390556) [ 8848.179943] Lustre: DEBUG MARKER: == sanity test 48b: Access removed working dir (should return errors)=========================================================== 05:22:45 (1787390565) [ 8856.540133] Lustre: DEBUG MARKER: == sanity test 48c: Access removed working subdir (should return errors) ========================================================== 05:22:54 (1787390574) [ 8864.950978] Lustre: DEBUG MARKER: == sanity test 48d: Access removed parent subdir (should return errors) ========================================================== 05:23:02 (1787390582) [ 8874.145608] Lustre: DEBUG MARKER: == sanity test 48e: Access to recreated parent subdir (should return errors) ========================================================== 05:23:12 (1787390592) [ 8882.066523] Lustre: DEBUG MARKER: == sanity test 48f: non-zero nlink dir unlink won't LBUG() ========================================================== 05:23:20 (1787390600) [ 8883.760319] Lustre: DEBUG MARKER: SKIP: sanity test_48f needs different host for mdt1 mdt2 [ 8885.474886] Lustre: DEBUG MARKER: == sanity test 49a: Change max_pages_per_rpc won't break osc extent ========================================================== 05:23:23 (1787390603) [ 9755.476741] Lustre: DEBUG MARKER: == sanity test 49b: verify max_mb_per_rpc_read/write after setting max_pages_per_rpc ========================================================== 05:37:53 (1787391473) [ 9763.943724] Lustre: DEBUG MARKER: == sanity test 49c: verify max_mb_per_rpc_read/write ===== 05:38:01 (1787391481) [ 9772.369630] Lustre: DEBUG MARKER: == sanity test 50: special situations: /proc symlinks ========================================================================= 05:38:10 (1787391490) [ 9780.655551] Lustre: DEBUG MARKER: == sanity test 51a: special situations: split htree with empty entry ============================================================ 05:38:18 (1787391498) [ 9796.721280] Lustre: DEBUG MARKER: == sanity test 51b: exceed 64k subdirectory nlink limit on create, verify unlink ========================================================== 05:38:34 (1787391514) [11913.998921] Lustre: DEBUG MARKER: SKIP: sanity test_51c skipping SLOW test 51c [11916.157348] Lustre: DEBUG MARKER: == sanity test 51d: check LOV round-robin OST object distribution ========================================================== 06:13:53 (1787393633) [11917.999283] Lustre: DEBUG MARKER: SKIP: sanity test_51d needs >= 3 OSTs [11920.063216] Lustre: DEBUG MARKER: SKIP: sanity test_51e skipping SLOW test 51e [11922.395625] Lustre: DEBUG MARKER: == sanity test 51f: check many open files limit ========== 06:13:59 (1787393639) [12237.912095] Lustre: DEBUG MARKER: == sanity test 52a: append-only flag test (should return errors) ========================================================== 06:19:15 (1787393955) [12246.384860] Lustre: DEBUG MARKER: == sanity test 52b: immutable flag test (should return errors) ================================================================= 06:19:24 (1787393964) [12254.654788] Lustre: DEBUG MARKER: == sanity test 53: verify that MDS and OSTs agree on pre-creation ============================================================== 06:19:32 (1787393972) [12269.065868] Lustre: DEBUG MARKER: == sanity test 54a: unix domain socket test ============== 06:19:46 (1787393986) [12277.282539] Lustre: DEBUG MARKER: == sanity test 54b: char device works in lustre ================================================================================ 06:19:55 (1787393995) [12285.316398] Lustre: DEBUG MARKER: == sanity test 54c: block device works in lustre =============================================================================== 06:20:03 (1787394003) [12294.852146] Lustre: DEBUG MARKER: == sanity test 54d: fifo device works in lustre ================================================================================ 06:20:12 (1787394012) [12301.805677] Lustre: DEBUG MARKER: == sanity test 54e: console/tty device works in lustre ================================================================================ 06:20:19 (1787394019) [12311.248763] Lustre: DEBUG MARKER: == sanity test 55a: OBD device life cycle unit tests ===== 06:20:28 (1787394028) [12319.773186] Lustre: DEBUG MARKER: == sanity test 55b: Load and unload max OBD devices ====== 06:20:37 (1787394037) [12995.375278] Lustre: DEBUG MARKER: == sanity test 55c: obd_mod_rpcs_test ==================== 06:31:52 (1787394712) [13005.262672] Lustre: DEBUG MARKER: == sanity test 56a: check /home/green/git/lustre-release/lustre/utils/lfs getstripe ========================================================== 06:32:03 (1787394723) [13015.895892] Lustre: DEBUG MARKER: == sanity test 56b: check /home/green/git/lustre-release/lustre/utils/lfs getdirstripe ========================================================== 06:32:14 (1787394734) [13024.670219] Lustre: DEBUG MARKER: == sanity test 56bb: check /home/green/git/lustre-release/lustre/utils/lfs getdirstripe layout is YAML ========================================================== 06:32:22 (1787394742) [13034.310748] Lustre: DEBUG MARKER: == sanity test 56bc: check '/home/green/git/lustre-release/lustre/utils/lfs getdirstripe --yaml' params are valid ========================================================== 06:32:32 (1787394752) [13044.906302] Lustre: DEBUG MARKER: == sanity test 56bd: check '/home/green/git/lustre-release/lustre/utils/lfs getdirstripe' output contains header only when needed ========================================================== 06:32:42 (1787394762) [13053.810661] Lustre: DEBUG MARKER: == sanity test 56c: check 'lfs df' showing device status ========================================================== 06:32:51 (1787394771) [13090.179730] Lustre: DEBUG MARKER: == sanity test 56ca: 'lfs df -v' correctly reports 'R' flag when OST set as Readonly ========================================================== 06:33:28 (1787394808) [13123.097422] Lustre: DEBUG MARKER: == sanity test 56d: 'lfs df -v' prints only configured devices ========================================================== 06:34:00 (1787394840) [13129.346361] Lustre: DEBUG MARKER: == sanity test 56e: 'lfs df' Handle non LustreFS [13136.197900] Lustre: DEBUG MARKER: == sanity test 56g: check lfs find -name ================= 06:34:14 (1787394854) [13145.025548] Lustre: DEBUG MARKER: == sanity test 56h: check lfs find ! -name =============== 06:34:23 (1787394863) [13153.432437] Lustre: DEBUG MARKER: == sanity test 56i: check 'lfs find -ost UUID' skips directories ========================================================== 06:34:31 (1787394871) [13160.998751] Lustre: DEBUG MARKER: == sanity test 56ib: check 'lfs find -ost INDEX_RANGE' command ========================================================== 06:34:38 (1787394878) [13168.697943] Lustre: DEBUG MARKER: == sanity test 56j: check lfs find -type d =============== 06:34:46 (1787394886) [13176.733651] Lustre: DEBUG MARKER: == sanity test 56k: check lfs find -type f =============== 06:34:54 (1787394894) [13184.060067] Lustre: DEBUG MARKER: == sanity test 56l: check lfs find -type b =============== 06:35:02 (1787394902) [13192.093549] Lustre: DEBUG MARKER: == sanity test 56m: check lfs find -type c =============== 06:35:10 (1787394910) [13200.760200] Lustre: DEBUG MARKER: == sanity test 56n: check lfs find -type l =============== 06:35:18 (1787394918) [13208.025705] Lustre: DEBUG MARKER: == sanity test 56o: check lfs find -mtime for old files == 06:35:25 (1787394925) [13219.080486] Lustre: DEBUG MARKER: == sanity test 56ob: check lfs find -atime -mtime -ctime with units ========================================================== 06:35:37 (1787394937) [13236.604465] Lustre: DEBUG MARKER: SKIP: sanity test_56oc skipping excluded test 56oc [13238.883718] Lustre: DEBUG MARKER: == sanity test 56od: check lfs find -btime with units ==== 06:35:56 (1787394956) [13258.124546] Lustre: DEBUG MARKER: == sanity test 56oe: check lfs find with time range ====== 06:36:16 (1787394976) [13396.229445] Lustre: DEBUG MARKER: == sanity test 56p: check lfs find -uid and ! -uid ======= 06:38:33 (1787395113) [13408.103481] Lustre: DEBUG MARKER: == sanity test 56q: check lfs find -gid and ! -gid ======= 06:38:45 (1787395125) [13419.058602] Lustre: DEBUG MARKER: == sanity test 56r: check lfs find -size works =========== 06:38:56 (1787395136) [13431.586521] Lustre: DEBUG MARKER: == sanity test 56ra: check lfs find -size -lazy works for data on OSTs ========================================================== 06:39:09 (1787395149) [13447.849515] Lustre: DEBUG MARKER: == sanity test 56rb: check lfs find --size --ost/--mdt works ========================================================== 06:39:26 (1787395166) [13459.275984] Lustre: DEBUG MARKER: == sanity test 56rc: check lfs find --mdt-count/--mdt-hash works ========================================================== 06:39:36 (1787395176) [13475.608670] Lustre: DEBUG MARKER: == sanity test 56rd: check lfs find --printf special files ========================================================== 06:39:53 (1787395193) [13485.462656] Lustre: DEBUG MARKER: == sanity test 56re: check lfs find -printf width format specifiers are consistant with regular find ========================================================== 06:40:02 (1787395202) [13499.758044] Lustre: DEBUG MARKER: == sanity test 56rf: check lfs find -printf width format specifiers for lustre specific formats ========================================================== 06:40:17 (1787395217) [13524.579536] Lustre: DEBUG MARKER: == sanity test 56s: check lfs find -stripe-count works === 06:40:42 (1787395242) [13538.212877] Lustre: DEBUG MARKER: == sanity test 56t: check lfs find -stripe-size works ==== 06:40:55 (1787395255) [13554.361990] Lustre: DEBUG MARKER: == sanity test 56u: check lfs find -stripe-index works === 06:41:11 (1787395271) [13567.919713] Lustre: DEBUG MARKER: == sanity test 56v: check 'lfs find -m match with lfs getstripe -m' ========================================================== 06:41:25 (1787395285) [13580.414169] Lustre: DEBUG MARKER: == sanity test 56wa: check 'lfs migrate -c stripe_count' works ========================================================== 06:41:38 (1787395298) [13865.063558] Lustre: DEBUG MARKER: == sanity test 56wb: check 'lfs migrate' pool support ==== 06:46:22 (1787395582) [13894.154586] Lustre: DEBUG MARKER: == sanity test 56wc: check unrecognized options for lfs migrate are passed through ========================================================== 06:46:51 (1787395611) [13928.289977] Lustre: DEBUG MARKER: == sanity test 56wd: check lfs_migrate --rsync and --no-rsync work ========================================================== 06:47:26 (1787395646) [13935.590590] Lustre: DEBUG MARKER: == sanity test 56we: check lfs migrate --non-direct|-D support ========================================================== 06:47:33 (1787395653) [13945.168545] Lustre: DEBUG MARKER: == sanity test 56x: lfs migration support ================ 06:47:42 (1787395662) [13954.362909] Lustre: DEBUG MARKER: == sanity test 56xB: lfs migrate with -0, --null, --files-from arguments ========================================================== 06:47:52 (1787395672) [13970.000887] Lustre: DEBUG MARKER: == sanity test 56xC: lfs migration can accept FID list file ========================================================== 06:48:07 (1787395687) [13983.503882] Lustre: DEBUG MARKER: == sanity test 56xD: lfs_migration can accept FIDs ======= 06:48:21 (1787395701) [13992.781859] Lustre: DEBUG MARKER: == sanity test 56xa: lfs migration --block support ======= 06:48:31 (1787395711) [14001.568473] Lustre: DEBUG MARKER: == sanity test 56xab: lfs migration --block on changing file ========================================================== 06:48:39 (1787395719) [14010.657078] Lustre: DEBUG MARKER: == sanity test 56xb: lfs migration hard link support ===== 06:48:48 (1787395728) [14306.947234] Lustre: DEBUG MARKER: == sanity test 56xc: lfs migration autostripe ============ 06:53:44 (1787396024) [14317.312706] Lustre: DEBUG MARKER: == sanity test 56xd: check lfs migrate --yaml and --copy support ========================================================== 06:53:54 (1787396034) [14341.419950] Lustre: DEBUG MARKER: == sanity test 56xe: migrate a composite layout file ===== 06:54:18 (1787396058) [14364.805647] Lustre: DEBUG MARKER: == sanity test 56xf: FID is not lost during migration of a composite layout file ========================================================== 06:54:42 (1787396082) [14380.214297] Lustre: DEBUG MARKER: == sanity test 56xg: lfs migrate pool support ============ 06:54:58 (1787396098) [14458.486141] Lustre: DEBUG MARKER: == sanity test 56xh: lfs migrate bandwidth limitation support ========================================================== 06:56:15 (1787396175) [14500.981615] Lustre: DEBUG MARKER: == sanity test 56xi: lfs migrate stats support =========== 06:56:58 (1787396218) [14516.490969] Lustre: DEBUG MARKER: == sanity test 56xj: lfs migrate -b should not cause starvation of threads on OSS ========================================================== 06:57:14 (1787396234) [14561.000676] Lustre: DEBUG MARKER: == sanity test 56xk: lfs mirror resync bandwidth limitation support ========================================================== 06:57:58 (1787396278) [14579.110945] Lustre: DEBUG MARKER: == sanity test 56xl: lfs mirror resync stats support ===== 06:58:16 (1787396296) [14593.641188] Lustre: DEBUG MARKER: == sanity test 56y: lfs find -L raid0|released =========== 06:58:31 (1787396311) [14601.780722] Lustre: DEBUG MARKER: == sanity test 56z: lfs find should continue after an error ========================================================== 06:58:39 (1787396319) [14614.021143] Lustre: DEBUG MARKER: == sanity test 56Aa: lfs find --size under striped dir === 06:58:52 (1787396332) [14673.353721] Lustre: DEBUG MARKER: == sanity test 56ab: lfs find --blocks =================== 06:59:50 (1787396390) [14689.769424] Lustre: DEBUG MARKER: == sanity test 56aca: check lfs find -perm with octal representation ========================================================== 07:00:07 (1787396407) [14743.719586] Lustre: DEBUG MARKER: == sanity test 56acb: check lfs find -perm with symbolic representation ========================================================== 07:01:01 (1787396461) [14763.247190] Lustre: DEBUG MARKER: == sanity test 56acc: check parsing error for lfs find -perm ========================================================== 07:01:21 (1787396481) [14772.619662] Lustre: DEBUG MARKER: == sanity test 56Ba: test lfs find --component-end, -start, -count, and -flags ========================================================== 07:01:29 (1787396489) [14794.101887] Lustre: DEBUG MARKER: == sanity test 56Ca: check lfs find --mirror-count|-N and --mirror-state ========================================================== 07:01:51 (1787396511) [14810.243949] Lustre: DEBUG MARKER: == sanity test 56Da: test lfs find with long paths ======= 07:02:07 (1787396527) [14836.370542] Lustre: DEBUG MARKER: == sanity test 56Db: test 'lfs df -m' only shows MDT devices ========================================================== 07:02:33 (1787396553) [14846.320953] Lustre: DEBUG MARKER: == sanity test 56Dc: test 'lfs df -o' only shows OST devices ========================================================== 07:02:43 (1787396563) [14856.504344] Lustre: DEBUG MARKER: == sanity test 56Dd: test lfs find with mindepth argument ========================================================== 07:02:53 (1787396573) [14865.995436] Lustre: DEBUG MARKER: == sanity test 56Ea: test lfs find -printf option ======== 07:03:03 (1787396583) [14900.495578] Lustre: DEBUG MARKER: == sanity test 56Eaa: test lfs find -printf added functions ========================================================== 07:03:37 (1787396617) [15190.861114] Lustre: DEBUG MARKER: == sanity test 56Eab: test lfs find -ls function ========= 07:08:28 (1787396908) [15219.290167] Lustre: DEBUG MARKER: == sanity test 56Eb: check lfs getstripe on symlink ====== 07:08:56 (1787396936) [15230.486519] Lustre: DEBUG MARKER: == sanity test 56Ebb: check /home/green/git/lustre-release/lustre/utils/lfs getdirstripe for FIFO file ========================================================== 07:09:07 (1787396947) [15239.248680] Lustre: DEBUG MARKER: == sanity test 56Ec: check lfs getstripe,setstripe --hex --yaml ========================================================== 07:09:16 (1787396956) [15248.185873] Lustre: DEBUG MARKER: == sanity test 56Ed: verify new YAML format is valid and back-compatible ========================================================== 07:09:25 (1787396965) [15276.476700] Lustre: DEBUG MARKER: == sanity test 56Eda: check lfs find --links ============= 07:09:53 (1787396993) [15286.421983] Lustre: DEBUG MARKER: == sanity test 56Edb: check lfs find --links for directory striped on multiple MDTs ========================================================== 07:10:03 (1787397003) [15295.621412] Lustre: DEBUG MARKER: == sanity test 56Ef: lfs find with multiple paths ======== 07:10:13 (1787397013) [15304.634257] Lustre: DEBUG MARKER: == sanity test 56Eg: lfs find -xattr ===================== 07:10:22 (1787397022) [15314.771414] Lustre: DEBUG MARKER: == sanity test 56Eh: check lfs find --skip =============== 07:10:32 (1787397032) [15372.876738] Lustre: DEBUG MARKER: == sanity test 56Ei: test lfs find --printf prints correct projid for special files ========================================================== 07:11:31 (1787397091) [15381.304555] Lustre: DEBUG MARKER: == sanity test 56Ej: lfs migration --non-block copy ====== 07:11:39 (1787397099) [15390.815919] Lustre: DEBUG MARKER: == sanity test 56Ek: Test lfs find error handling with LLAPI_FAIL_LOC ========================================================== 07:11:48 (1787397108) [15402.584687] Lustre: DEBUG MARKER: == sanity test 57a: verify MDS filesystem created with large inodes ============================================================ 07:12:00 (1787397120) [15414.823620] Lustre: DEBUG MARKER: == sanity test 57b: default LOV EAs are stored inside large inodes ============================================================= 07:12:12 (1787397132) [15439.874927] Lustre: DEBUG MARKER: == sanity test 58: verify cross-platform wire constants ======================================================================== 07:12:37 (1787397157) [15447.975670] Lustre: DEBUG MARKER: == sanity test 59: verify cancellation of llog records async =================================================================== 07:12:45 (1787397165) [15500.185383] Lustre: DEBUG MARKER: == sanity test complete, duration 15188 sec ============== 07:13:37 (1787397217) [15502.146546] Lustre: DEBUG MARKER: === sanity: start cleanup 07:13:39 (1787397219) === [15709.248240] Lustre: DEBUG MARKER: === sanity: finish cleanup 07:17:06 (1787397426) === [15711.774597] Lustre: Failing over lustre-MDT0000 [15712.750799] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [15712.761660] LustreError: Skipped 1 previous similar message [15712.764487] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15712.779455] Lustre: Skipped 2 previous similar messages [15712.786484] LustreError: 23004:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15712.804209] LustreError: 23004:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 13 previous similar messages [15713.179118] Lustre: server umount lustre-MDT0000 complete [15718.374042] LustreError: 6513:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15718.397950] LustreError: 6513:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [15728.612730] LustreError: 6514:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15728.631860] LustreError: 6514:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [15730.143739] Lustre: 3637:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787397432/real 1787397432] req@ffff9da476462d80 x1874205713210880/t0(0) o400->MGC192.168.202.149@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787397448 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15730.184143] Lustre: 3637:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [15730.204781] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15743.793893] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15745.063546] LustreError: 6508:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.49@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [15745.085033] LustreError: 6508:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 15 previous similar messages [15756.454793] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15756.463190] Lustre: Skipped 1 previous similar message [15756.562613] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [15756.577213] Lustre: Skipped 1 previous similar message [15757.243349] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [15757.255690] Lustre: Skipped 1 previous similar message [15761.917374] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [15761.937395] Lustre: Skipped 3 previous similar messages [15762.191354] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [15762.258443] Lustre: lustre-MDT0000: Recovery over after 0:05, of 2 clients 2 recovered and 0 were evicted. [15762.302496] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000403:7616 to 0x2c0000403:7681) [15762.317240] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000403:7699 to 0x280000403:7745) [15773.197826] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [15775.312915] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [15782.882036] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [15782.893075] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15782.908861] Lustre: Skipped 3 previous similar messages [15782.919424] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15782.922197] Lustre: Skipped 2 previous similar messages [15787.489166] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15787.505568] Lustre: Skipped 2 previous similar messages [15789.248667] Lustre: server umount lustre-MDT0000 complete [15792.609028] LustreError: 11761:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15792.634769] LustreError: 11761:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 17 previous similar messages [15798.345417] LustreError: 27330:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787397517 with bad export cookie 7446420174715319154 [15798.355419] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15798.888559] Lustre: server umount lustre-MDT0001 complete [15818.080556] Lustre: 3636:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787397521/real 1787397521] req@ffff9da34c7cca80 x1874205713277952/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787397537 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15818.107295] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15818.123329] Lustre: Skipped 3 previous similar messages [15819.296949] Lustre: server umount lustre-OST0000 complete [15820.260799] Lustre: 3637:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787397523/real 1787397523] req@ffff9da47d8f9f80 x1874205713278208/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787397539 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15824.290573] Lustre: 3636:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787397527/real 1787397527] req@ffff9da47d8f9c00 x1874205713278464/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787397543 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15829.471164] Lustre: 3637:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787397532/real 1787397532] req@ffff9da45d157b80 x1874205713278848/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787397548 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15830.118764] Lustre: server umount lustre-OST0001 complete [15850.724972] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [15854.143932] Key type lgssc unregistered [15854.488870] LNet: 140121:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [15854.496910] LNetError: 140121:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [15854.525865] LNet: Removed LNI 192.168.202.149@tcp [15855.698307] Key type .llcrypt unregistered [15855.703276] Key type ._llcrypt unregistered