[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 462382210 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002410] x2apic enabled [ 0.003008] Switched APIC routing to physical x2apic. [ 0.004014] kvm-guest: setup PV IPIs [ 0.007575] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009013] pid_max: default: 32768 minimum: 301 [ 0.010136] LSM: Security Framework initializing [ 0.011056] Yama: becoming mindful. [ 0.012035] SELinux: Initializing. [ 0.013077] *** VALIDATE selinux *** [ 0.021367] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025708] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026120] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027108] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029102] *** VALIDATE tmpfs *** [ 0.031260] *** VALIDATE proc *** [ 0.033213] *** VALIDATE cgroup *** [ 0.034011] *** VALIDATE cgroup2 *** [ 0.035264] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037064] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039029] Spectre V2 : User space: Vulnerable [ 0.040027] Speculative Store Bypass: Vulnerable [ 0.043510] debug: unmapping init [mem 0xffffffffb4259000-0xffffffffb4260fff] [ 0.045162] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046485] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047024] ... version: 2 [ 0.048011] ... bit width: 48 [ 0.049013] ... generic registers: 4 [ 0.050013] ... value mask: 0000ffffffffffff [ 0.051014] ... max period: 00007fffffffffff [ 0.052014] ... fixed-purpose events: 3 [ 0.053011] ... event mask: 000000070000000f [ 0.054277] rcu: Hierarchical SRCU implementation. [ 0.056423] smp: Bringing up secondary CPUs ... [ 0.057607] x86: Booting SMP configuration: [ 0.058027] .... node #0, CPUs: #1 #2 #3 [ 0.062708] smp: Brought up 1 node, 4 CPUs [ 0.064020] smpboot: Max logical packages: 1 [ 0.065014] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.175071] node 0 deferred pages initialised in 108ms [ 0.179215] devtmpfs: initialized [ 0.182349] x86/mm: Memory block size: 128MB [ 0.186081] gcov: version magic: 0x41383552 [ 0.190247] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.195097] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.197475] pinctrl core: initialized pinctrl subsystem [ 0.200208] [ 0.201008] ************************************************************* [ 0.205013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.209014] ** ** [ 0.212011] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.215017] ** ** [ 0.219015] ** This means that this kernel is built to expose internal ** [ 0.222014] ** IOMMU data structures, which may compromise security on ** [ 0.225020] ** your system. ** [ 0.229012] ** ** [ 0.231012] ** If you see this message and you are not debugging the ** [ 0.233013] ** kernel, report this immediately to your vendor! ** [ 0.236014] ** ** [ 0.239014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.241021] ************************************************************* [ 0.244417] NET: Registered protocol family 16 [ 0.246636] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.249065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.253073] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.258104] cpuidle: using governor menu [ 0.259600] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.262431] PCI: Using configuration type 1 for base access [ 0.267150] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.281149] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.282024] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.284029] cryptd: max_cpu_qlen set to 1000 [ 0.286345] ACPI: Added _OSI(Module Device) [ 0.289093] ACPI: Added _OSI(Processor Device) [ 0.291023] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.293018] ACPI: Added _OSI(Processor Aggregator Device) [ 0.300576] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.309686] ACPI: Interpreter enabled [ 0.311064] ACPI: PM: (supports S0 S3 S4 S5) [ 0.313013] ACPI: Using IOAPIC for interrupt routing [ 0.314146] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.319464] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.330456] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.332044] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.335019] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.338072] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.343540] acpiphp: Slot [2] registered [ 0.345194] acpiphp: Slot [5] registered [ 0.347121] acpiphp: Slot [6] registered [ 0.349131] acpiphp: Slot [7] registered [ 0.350108] acpiphp: Slot [8] registered [ 0.352092] acpiphp: Slot [9] registered [ 0.353077] acpiphp: Slot [10] registered [ 0.355105] acpiphp: Slot [3] registered [ 0.356088] acpiphp: Slot [4] registered [ 0.358096] acpiphp: Slot [11] registered [ 0.359112] acpiphp: Slot [12] registered [ 0.361101] acpiphp: Slot [13] registered [ 0.363177] acpiphp: Slot [14] registered [ 0.365099] acpiphp: Slot [15] registered [ 0.367092] acpiphp: Slot [16] registered [ 0.368097] acpiphp: Slot [17] registered [ 0.370131] acpiphp: Slot [18] registered [ 0.372093] acpiphp: Slot [19] registered [ 0.373087] acpiphp: Slot [20] registered [ 0.374088] acpiphp: Slot [21] registered [ 0.376092] acpiphp: Slot [22] registered [ 0.378090] acpiphp: Slot [23] registered [ 0.379083] acpiphp: Slot [24] registered [ 0.380106] acpiphp: Slot [25] registered [ 0.382088] acpiphp: Slot [26] registered [ 0.383126] acpiphp: Slot [27] registered [ 0.385142] acpiphp: Slot [28] registered [ 0.387126] acpiphp: Slot [29] registered [ 0.389175] acpiphp: Slot [30] registered [ 0.391121] acpiphp: Slot [31] registered [ 0.392055] PCI host bridge to bus 0000:00 [ 0.393019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.396021] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.398020] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.400034] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.403022] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.405030] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.407154] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.409907] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.413409] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.429018] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.433010] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.435062] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.438023] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.440026] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.443697] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.444796] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.451058] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.455000] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.462021] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.477021] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.483022] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.491323] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.501015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.508016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.523016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.534000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.546022] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.553020] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.575021] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.586606] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.593017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.601016] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.619017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.628799] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.637020] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.649035] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.670025] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.688000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.695021] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.700020] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.718025] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.732549] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.739021] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.747016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.760014] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.772310] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.774288] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.776338] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.778650] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.781410] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.786062] iommu: Default domain type: Passthrough [ 0.787414] SCSI subsystem initialized [ 0.789109] ACPI: bus type USB registered [ 0.790109] usbcore: registered new interface driver usbfs [ 0.792075] usbcore: registered new interface driver hub [ 0.793070] usbcore: registered new device driver usb [ 0.794448] pps_core: LinuxPPS API ver. 1 registered [ 0.796013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.798052] PTP clock support registered [ 0.800085] EDAC MC: Ver: 3.0.0 [ 0.803081] PCI: Using ACPI for IRQ routing [ 0.804526] NetLabel: Initializing [ 0.807028] NetLabel: domain hash size = 128 [ 0.809081] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.812313] NetLabel: unlabeled traffic allowed by default [ 0.816087] vgaarb: loaded [ 0.817372] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.822085] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.829576] clocksource: Switched to clocksource kvm-clock [ 0.940417] VFS: Disk quotas dquot_6.6.0 [ 0.941513] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.943343] *** VALIDATE ramfs *** [ 0.944284] *** VALIDATE hugetlbfs *** [ 0.946346] pnp: PnP ACPI init [ 0.948681] pnp: PnP ACPI: found 6 devices [ 0.966726] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.969831] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.971861] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.974346] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.977407] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.979927] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.983270] NET: Registered protocol family 2 [ 0.986140] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.992865] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.997244] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.004572] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.008953] TCP: Hash tables configured (established 65536 bind 65536) [ 1.012873] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.016606] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.019896] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.023326] NET: Registered protocol family 1 [ 1.027748] RPC: Registered named UNIX socket transport module. [ 1.030846] RPC: Registered udp transport module. [ 1.032378] RPC: Registered tcp transport module. [ 1.034489] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.037180] NET: Registered protocol family 44 [ 1.039415] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.041661] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.044151] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.046753] PCI: CLS 0 bytes, default 64 [ 1.048804] Unpacking initramfs... [ 2.432222] debug: unmapping init [mem 0xffff8bc27cc54000-0xffff8bc27ffbffff] [ 2.436099] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.437857] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.439911] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.928331] Initialise system trusted keyrings [ 2.930358] Key type blacklist registered [ 2.931955] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.941152] zbud: loaded [ 2.944772] *** VALIDATE nfs *** [ 2.946095] *** VALIDATE nfs4 *** [ 2.947940] pstore: using deflate compression [ 2.951911] Platform Keyring initialized [ 3.069448] NET: Registered protocol family 38 [ 3.070704] Key type asymmetric registered [ 3.071914] Asymmetric key parser 'x509' registered [ 3.073306] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.076593] io scheduler mq-deadline registered [ 3.078405] io scheduler kyber registered [ 3.082349] io scheduler bfq registered [ 3.086938] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.092351] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.098132] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.103567] ACPI: Power Button [PWRF] [ 3.109839] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.119098] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.136803] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.145376] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.166253] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.196503] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.226107] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.232684] Non-volatile memory driver v1.3 [ 3.234261] Linux agpgart interface v0.103 [ 3.264844] virtio_blk virtio1: [vda] 146144 512-byte logical blocks (74.8 MB/71.4 MiB) [ 3.267420] vda: detected capacity change from 0 to 74825728 [ 3.287340] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.292052] vdb: detected capacity change from 0 to 1073741824 [ 3.309833] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.312218] vdc: detected capacity change from 0 to 2621440000 [ 3.324588] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.327684] vdd: detected capacity change from 0 to 2621440000 [ 3.340508] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.343128] vde: detected capacity change from 0 to 4294967296 [ 3.356400] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.359918] vdf: detected capacity change from 0 to 4294967296 [ 3.365962] libphy: Fixed MDIO Bus: probed [ 3.370811] usbcore: registered new interface driver usbserial_generic [ 3.373391] usbserial: USB Serial support registered for generic [ 3.375925] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.380507] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.382616] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.385367] mousedev: PS/2 mouse device common for all mice [ 3.389808] rtc_cmos 00:05: RTC can wake from S4 [ 3.392263] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.393634] rtc_cmos 00:05: registered as rtc0 [ 3.403349] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.405884] intel_pstate: CPU model not supported [ 3.409483] hid: raw HID events driver (C) Jiri Kosina [ 3.412178] usbcore: registered new interface driver usbhid [ 3.415071] usbhid: USB HID core driver [ 3.417161] drop_monitor: Initializing network drop monitor service [ 3.418074] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.423943] Initializing XFRM netlink socket [ 3.428936] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.429706] NET: Registered protocol family 10 [ 3.436989] Segment Routing with IPv6 [ 3.438649] NET: Registered protocol family 17 [ 3.440856] mpls_gso: MPLS GSO support [ 3.445758] RAS: Correctable Errors collector initialized. [ 3.448233] AVX version of gcm_enc/dec engaged. [ 3.449921] AES CTR mode by8 optimization enabled [ 3.526434] sched_clock: Marking stable (3526407755, 0)->(4492293486, -965885731) [ 3.531489] registered taskstats version 1 [ 3.533569] Loading compiled-in X.509 certificates [ 3.535707] zswap: loaded using pool lzo/zbud [ 3.562674] Key type big_key registered [ 3.581558] Key type encrypted registered [ 3.583943] ima: No TPM chip found, activating TPM-bypass! [ 3.587696] ima: Allocated hash algorithm: sha1 [ 3.590200] ima: No architecture policies found [ 3.593266] evm: Initialising EVM extended attributes: [ 3.595118] evm: security.selinux [ 3.596362] evm: security.ima [ 3.597396] evm: security.capability [ 3.598817] evm: HMAC attrs: 0x1 [ 3.601202] rtc_cmos 00:05: setting system clock to 2026-08-25 21:08:25 UTC (1787692105) [ 3.607878] debug: unmapping init [mem 0xffffffffb5203000-0xffffffffb53fffff] [ 3.611519] debug: unmapping init [mem 0xffffffffb3f82000-0xffffffffb4258fff] [ 3.620081] Write protecting the kernel read-only data: 28672k [ 3.622775] debug: unmapping init [mem 0xffffffffb2603000-0xffffffffb27fffff] [ 3.624710] debug: unmapping init [mem 0xffffffffb2f14000-0xffffffffb2ffffff] [ 3.660637] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.670167] systemd[1]: Detected virtualization kvm. [ 3.672263] systemd[1]: Detected architecture x86-64. [ 3.674632] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.701437] systemd[1]: No hostname configured. [ 3.703771] systemd[1]: Set hostname to . [ 3.706434] random: systemd: uninitialized urandom read (16 bytes read) [ 3.709424] systemd[1]: Initializing machine ID from random generator. [ 3.851348] random: systemd: uninitialized urandom read (16 bytes read) [ 3.854322] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.862395] random: systemd: uninitialized urandom read (16 bytes read) [ 3.865433] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.871115] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Slices. [ OK ] Reached target Local File Systems. [ OK ] Reached target Swap. [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Journal Service... Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Sockets. Starting Setup Virtual Console... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.777857] device-mapper: uevent: version 1.0.3 [ 4.780631] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.532487] random: fast init done [ 5.537834] virtio_net virtio0 ens2: renamed from eth0 [ 5.601812] scsi host0: ata_piix [ 5.630205] scsi host1: ata_piix [ 5.632084] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.635023] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.256656] random: crng init done [ 10.260108] random: 7 urandom warning(s) missed due to ratelimiting [ 10.563242] dracut-initqueue[588]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.335933] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 13.247859] printk: systemd: 26 output lines suppressed due to ratelimiting [ 13.530554] SELinux: Disabled at runtime. [ 13.587824] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 13.596252] systemd[1]: Detected virtualization kvm. [ 13.598129] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 14.159834] systemd[1]: initrd-switch-root.service: Succeeded. [ 14.165106] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 14.171685] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 14.179729] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 14.183303] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 14.193672] systemd[1]: Starting Journal Service... Starting Journal Service... [ 14.203124] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice User and Session Slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [ OK ] Stopped target Switch Root. [ OK ] Reached target rpc_pipefs.target. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Mounting POSIX Message Queue File System... Mounting Kernel Debug File System... [ OK ] Stopped target Initrd Root File System. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-getty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Initrd File Systems. [ 14.778382] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 15.127776] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 15.515517] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 15.571269] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 15.775317] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 15.924774] EDAC sbridge: Ver: 1.1.2 [ 18.154147] Key type dns_resolver registered [ 18.473741] NFS: Registering the id_resolver key type [ 18.476109] Key type id_resolver registered [ 18.477705] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. [ OK ] Started Authorization Manager. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg323-server login: [ 47.200168] libcfs: loading out-of-tree module taints kernel. [ 47.222439] Key type ._llcrypt registered [ 47.225262] Key type .llcrypt registered [ 47.276313] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_hostid [ 60.691465] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing load_modules_local [ 61.944823] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 61.973447] alg: No test for adler32 (adler32-zlib) [ 63.244579] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 64.199959] LNet: Added LNI 192.168.203.123@tcp [8/256/0/180] [ 65.943201] Key type lgssc registered [ 68.246731] Lustre: Echo OBD driver; http://www.lustre.org/ [ 74.442330] hrtimer: interrupt took 2713821 ns [ 87.816707] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 125.511558] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing load_modules_local [ 137.019040] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 137.044107] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 138.338372] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 138.421456] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 138.567101] Lustre: lustre-MDT0000: new disk, initializing [ 138.698422] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 138.731515] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 143.286930] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 155.582963] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 155.662377] Lustre: 6495:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 155.719675] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 155.731460] Lustre: Skipped 1 previous similar message [ 155.797596] Lustre: lustre-MDT0001: new disk, initializing [ 155.848843] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 155.876344] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 155.893572] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 159.904957] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 163.913212] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 171.687219] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 171.900623] Lustre: lustre-OST0000: new disk, initializing [ 171.906495] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 171.910351] Lustre: 8429:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 171.949054] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 174.703121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 174.708931] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 174.762704] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 176.970568] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 189.477401] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 189.626903] Lustre: lustre-OST0001: new disk, initializing [ 189.630306] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 189.635708] Lustre: 9501:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 189.688600] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 195.485254] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 196.725986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 196.747147] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 196.854689] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 207.083655] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 214.941728] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 221.049900] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing check_logdir /tmp/testlogs/ [ 228.738565] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing yml_node [ 234.307921] Lustre: DEBUG MARKER: Client: 2.17.57.80 [ 237.224680] Lustre: DEBUG MARKER: MDS: 2.17.57.80 [ 239.866707] Lustre: DEBUG MARKER: OSS: 2.17.57.80 [ 241.623515] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity ============----- Tue Aug 25 17:12:21 EDT 2026 [ 258.320964] Lustre: DEBUG MARKER: - need MDS1_VERSION <= 2.16.61-1-g89cf292a8c2 (34683216 <= 34618625) for LU-18938, skip 360 [ 259.962839] Lustre: DEBUG MARKER: - need MDS1_VERSION < v2_14_55-100-g8a84c7f9c7 (34683216 < 34486116) for LU-14927, skip 0f [ 261.680590] Lustre: DEBUG MARKER: - need OST1_VERSION < v2_17_50-410-gbd8e7b9555 (34683216 < 34681754) for LU-12550, skip 216 [ 263.375238] Lustre: DEBUG MARKER: excepting tests: 56oc 42a 42c 42b 118c 118d 407 119i 216 817 411a [ 265.035413] Lustre: DEBUG MARKER: skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842 51c 51e 834 [ 266.520768] Lustre: DEBUG MARKER: === sanity: start setup 17:12:46 (1787692366) === [ 272.501569] Lustre: DEBUG MARKER: oleg323-client.virtnet: executing check_config_client /mnt/lustre [ 290.226439] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 293.648392] Lustre: 13505:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 297.713174] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 303.110794] Lustre: DEBUG MARKER: === sanity: finish setup 17:13:23 (1787692403) === [ 305.137838] Lustre: DEBUG MARKER: == sanity test 0a: touch; rm ============================= 17:13:25 (1787692405) [ 312.005403] Lustre: DEBUG MARKER: == sanity test 0b: chmod 0755 /mnt/lustre ======================================================================================= 17:13:31 (1787692411) [ 318.391701] Lustre: DEBUG MARKER: == sanity test 0c: check import proc ===================== 17:13:38 (1787692418) [ 323.765577] Lustre: DEBUG MARKER: == sanity test 0d: check export proc ===================== 17:13:43 (1787692423) [ 334.341724] Lustre: DEBUG MARKER: == sanity test 0e: Enable DNE MDT balancing for mkdir in the ROOT ========================================================== 17:13:54 (1787692434) [ 339.768291] Lustre: DEBUG MARKER: == sanity test 1: mkdir; remkdir; rmdir ================== 17:13:59 (1787692439) [ 344.960414] Lustre: DEBUG MARKER: == sanity test 2: mkdir; touch; rmdir; check file ======== 17:14:05 (1787692445) [ 350.768506] Lustre: DEBUG MARKER: == sanity test 3: mkdir; touch; rmdir; check dir ========= 17:14:10 (1787692450) [ 357.217082] Lustre: DEBUG MARKER: == sanity test 4: mkdir; touch dir/file; rmdir; checkdir (expect error) ========================================================== 17:14:17 (1787692457) [ 364.965044] Lustre: DEBUG MARKER: == sanity test 5: mkdir .../d5 .../d5/d2; chmod .../d5/d2 ========================================================== 17:14:24 (1787692464) [ 373.157345] Lustre: DEBUG MARKER: == sanity test 6a: touch f6a; chmod f6a; runas -u 500 -g 500 chmod f6a (should return error) ============================================================ 17:14:32 (1787692472) [ 380.798529] Lustre: DEBUG MARKER: == sanity test 6c: touch f6c; chown f6c; runas -u 500 -g 500 chown f6c (should return error) ============================================================ 17:14:40 (1787692480) [ 388.079884] Lustre: DEBUG MARKER: == sanity test 6e: touch+chgrp ; runas -u 500 -g 500 chgrp (should return error) ========================================================== 17:14:47 (1787692487) [ 394.714691] Lustre: DEBUG MARKER: == sanity test 6g: verify new dir in sgid dir inherits group ========================================================== 17:14:54 (1787692494) [ 402.329844] Lustre: DEBUG MARKER: == sanity test 6h: runas -u 500 -g 500 chown RUNAS_ID.0 .../ (should return error) ========================================================== 17:15:01 (1787692501) [ 409.512796] Lustre: DEBUG MARKER: == sanity test 6i: touch+chmod+chgrp ; chgrp read-only file should succeed ========================================================== 17:15:08 (1787692508) [ 416.072224] Lustre: DEBUG MARKER: == sanity test 7a: mkdir .../d7; mcreate .../d7/f; chmod .../d7/f ============================================================== 17:15:16 (1787692516) [ 422.652879] Lustre: DEBUG MARKER: == sanity test 7b: mkdir .../d7; mcreate d7/f2; echo foo > d7/f2 =============================================================== 17:15:22 (1787692522) [ 428.598401] Lustre: DEBUG MARKER: == sanity test 8: mkdir .../d8; touch .../d8/f; chmod .../d8/f ================================================================= 17:15:28 (1787692528) [ 434.325424] Lustre: DEBUG MARKER: == sanity test 9: mkdir .../d9 .../d9/d2 .../d9/d2/d3 ========================================================================== 17:15:34 (1787692534) [ 440.970368] Lustre: DEBUG MARKER: == sanity test 10: mkdir .../d10 .../d10/d2; touch .../d10/d2/f ================================================================ 17:15:40 (1787692540) [ 447.822751] Lustre: DEBUG MARKER: == sanity test 11: mkdir .../d11 d11/d2; chmod .../d11/d2 ====================================================================== 17:15:47 (1787692547) [ 454.624281] Lustre: DEBUG MARKER: == sanity test 12: touch .../d12/f; chmod .../d12/f .../d12/f ================================================================== 17:15:54 (1787692554) [ 462.142894] Lustre: DEBUG MARKER: == sanity test 13: creat .../d13/f; dd .../d13/f; > .../d13/f ================================================================== 17:16:01 (1787692561) [ 468.946538] Lustre: DEBUG MARKER: == sanity test 14: touch .../d14/f; rm .../d14/f; rm .../d14/f ================================================================= 17:16:08 (1787692568) [ 475.266436] Lustre: DEBUG MARKER: == sanity test 15: touch .../d15/f; mv .../d15/f .../d15/f2 ==================================================================== 17:16:15 (1787692575) [ 482.197190] Lustre: DEBUG MARKER: == sanity test 16: touch .../d16/f; rm -rf .../d16/f ===== 17:16:22 (1787692582) [ 488.660299] Lustre: DEBUG MARKER: == sanity test 17a: symlinks: create, remove (real) ====== 17:16:28 (1787692588) [ 494.932286] Lustre: DEBUG MARKER: == sanity test 17b: symlinks: create, remove (dangling) == 17:16:34 (1787692594) [ 501.547525] Lustre: DEBUG MARKER: == sanity test 17c: symlinks: open dangling (should return error) ========================================================== 17:16:41 (1787692601) [ 507.920717] Lustre: DEBUG MARKER: == sanity test 17d: symlinks: create dangling ============ 17:16:48 (1787692608) [ 514.261354] Lustre: DEBUG MARKER: == sanity test 17e: symlinks: create recursive symlink (should return error) ========================================================== 17:16:54 (1787692614) [ 520.989746] Lustre: DEBUG MARKER: == sanity test 17f: symlinks: long and very long symlink name ========================================================== 17:17:00 (1787692620) [ 527.899932] Lustre: DEBUG MARKER: == sanity test 17g: symlinks: really long symlink name and inode boundaries ========================================================== 17:17:07 (1787692627) [ 536.085417] Lustre: DEBUG MARKER: == sanity test 17h: create objects: lov_free_memmd() doesn't lbug ========================================================== 17:17:16 (1787692636) [ 544.284968] Lustre: DEBUG MARKER: == sanity test 17i: don't panic on short symlink (should return error) ========================================================== 17:17:24 (1787692644) [ 545.970774] Lustre: *** cfs_fail_loc=143, val=0*** [ 552.412649] Lustre: DEBUG MARKER: == sanity test 17k: symlinks: rsync with xattrs enabled == 17:17:32 (1787692652) [ 561.134504] Lustre: DEBUG MARKER: == sanity test 17l: Ensure lgetxattr's returned xattr size is consistent ========================================================== 17:17:40 (1787692660) [ 568.448236] Lustre: DEBUG MARKER: == sanity test 17m: run e2fsck against MDT which contains short/long symlink ========================================================== 17:17:48 (1787692668) [ 644.565408] Lustre: Failing over lustre-MDT0001 [ 644.809826] Lustre: server umount lustre-MDT0001 complete [ 647.135478] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 647.152470] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 647.655175] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 647.673164] Lustre: Skipped 1 previous similar message [ 652.664492] LustreError: 16756:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.23@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 652.679112] LustreError: 16756:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 655.585236] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 656.145913] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 656.194237] Lustre: lustre-MDT0001: in recovery but waiting for the first client to connect [ 657.312682] Lustre: lustre-MDT0001: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 660.766339] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 661.486360] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 661.541246] Lustre: lustre-MDT0001: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 661.590118] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:9 to 0x280000400:33) [ 661.590975] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:7 to 0x2c0000400:33) [ 690.641059] Lustre: DEBUG MARKER: == sanity test 17n: run e2fsck against master/slave MDT which contains remote dir ========================================================== 17:19:50 (1787692790) [ 697.106778] Lustre: Failing over lustre-MDT0000 [ 697.321937] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 697.325423] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 697.335236] Lustre: Skipped 1 previous similar message [ 697.336335] LustreError: 6500:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 697.360566] LustreError: 6500:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 697.495219] Lustre: server umount lustre-MDT0000 complete [ 698.780385] LustreError: 10078:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.23@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 698.799927] LustreError: 10078:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 702.436780] LustreError: 6502:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 702.468515] LustreError: 6502:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 706.913034] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 707.089131] LustreError: MGC192.168.203.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 707.365143] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 707.454553] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 708.982465] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 711.907696] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 712.687406] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 712.699925] Lustre: Skipped 2 previous similar messages [ 712.740023] Lustre: lustre-MDT0000: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 712.784462] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:45 to 0x280000401:65) [ 712.784534] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:44 to 0x2c0000401:65) [ 715.410472] Lustre: Failing over lustre-MDT0001 [ 715.714871] Lustre: server umount lustre-MDT0001 complete [ 717.791953] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 717.801946] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 717.812371] LustreError: 6501:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 717.818969] Lustre: Skipped 4 previous similar messages [ 717.846252] LustreError: 6501:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 725.151735] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 725.530720] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 725.627355] Lustre: lustre-MDT0001: in recovery but waiting for the first client to connect [ 727.239379] Lustre: lustre-MDT0001: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 730.609659] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 730.625162] Lustre: Skipped 3 previous similar messages [ 730.668984] Lustre: lustre-MDT0001: Recovery over after 0:03, of 2 clients 2 recovered and 0 were evicted. [ 730.721211] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:49 to 0x280000400:65) [ 730.722774] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:49 to 0x2c0000400:65) [ 731.070600] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 740.267816] Lustre: Failing over lustre-MDT0000 [ 740.552255] LustreError: 25339:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (ffff8bc1c8e87c00) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 740.832840] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 740.844576] LustreError: 10078:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 740.852488] Lustre: Skipped 2 previous similar messages [ 740.855900] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 740.871375] LustreError: 10078:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 742.651693] Lustre: server umount lustre-MDT0000 complete [ 752.894354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 752.975320] LustreError: MGC192.168.203.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 753.329906] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 753.399331] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 754.207880] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 757.217915] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 758.768072] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 758.778889] Lustre: Skipped 2 previous similar messages [ 758.809097] Lustre: lustre-MDT0000: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 758.872859] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:45 to 0x280000401:97) [ 758.875356] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:44 to 0x2c0000401:97) [ 761.019287] Lustre: Failing over lustre-MDT0001 [ 761.353766] Lustre: server umount lustre-MDT0001 complete [ 763.364252] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 763.382238] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 763.399664] Lustre: Skipped 1 previous similar message [ 763.408537] LustreError: 8019:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 763.432648] LustreError: 8019:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 771.288215] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 771.689956] Lustre: lustre-MDT0001: in recovery but waiting for the first client to connect [ 773.719197] Lustre: lustre-MDT0001: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 776.068527] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 776.687291] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 776.695456] Lustre: Skipped 3 previous similar messages [ 776.881308] Lustre: lustre-MDT0001: Recovery over after 0:03, of 2 clients 2 recovered and 0 were evicted. [ 776.914636] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:49 to 0x2c0000400:97) [ 776.917958] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:49 to 0x280000400:97) [ 794.863595] Lustre: Failing over lustre-MDT0000 [ 795.388345] Lustre: server umount lustre-MDT0000 complete [ 797.168564] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 797.206370] Lustre: Skipped 2 previous similar messages [ 797.206901] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 797.224538] LustreError: 27303:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 797.224549] LustreError: 27303:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 807.493774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 807.669799] LustreError: MGC192.168.203.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 808.141390] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 808.149416] Lustre: Skipped 1 previous similar message [ 808.249745] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 809.340486] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 812.496389] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 813.551739] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 813.565132] Lustre: Skipped 2 previous similar messages [ 813.705798] Lustre: lustre-MDT0000: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [ 813.756025] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:123 to 0x2c0000401:161) [ 813.761231] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:123 to 0x280000401:161) [ 815.701160] Lustre: Failing over lustre-MDT0001 [ 816.117819] Lustre: server umount lustre-MDT0001 complete [ 818.146749] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 825.364871] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 825.736634] Lustre: lustre-MDT0001: in recovery but waiting for the first client to connect [ 826.902260] Lustre: lustre-MDT0001: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 829.172279] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 830.954206] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 830.963200] Lustre: Skipped 3 previous similar messages [ 831.001719] Lustre: lustre-MDT0001: Recovery over after 0:05, of 2 clients 2 recovered and 0 were evicted. [ 831.041218] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:123 to 0x280000400:161) [ 831.041566] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:123 to 0x2c0000400:161) [ 836.062489] Lustre: Failing over lustre-MDT0000 [ 836.077304] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 836.086995] Lustre: Skipped 9 previous similar messages [ 836.104468] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 836.308105] Lustre: server umount lustre-MDT0000 complete [ 846.194482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 846.412930] LustreError: MGC192.168.203.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 851.323870] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 852.094690] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:123 to 0x2c0000401:193) [ 852.101222] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:123 to 0x280000401:193) [ 854.858767] Lustre: Failing over lustre-MDT0001 [ 855.134555] Lustre: server umount lustre-MDT0001 complete [ 862.180048] LustreError: 21934:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 862.198199] LustreError: 21934:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 37 previous similar messages [ 865.995220] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 866.512422] Lustre: lustre-MDT0001: in recovery but waiting for the first client to connect [ 866.525362] Lustre: Skipped 1 previous similar message [ 867.615176] Lustre: lustre-MDT0001: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 867.622828] Lustre: Skipped 1 previous similar message [ 871.042959] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 871.928930] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 871.938850] Lustre: Skipped 6 previous similar messages [ 872.006944] Lustre: lustre-MDT0001: Recovery over after 0:05, of 2 clients 2 recovered and 0 were evicted. [ 872.018973] Lustre: Skipped 1 previous similar message [ 872.066089] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:123 to 0x2c0000400:193) [ 872.068232] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:123 to 0x280000400:193) [ 878.315346] Lustre: DEBUG MARKER: == sanity test 17o: stat file with incompat LMA feature == 17:22:58 (1787692978) [ 880.997522] Lustre: Failing over lustre-MDT0000 [ 881.261715] Lustre: server umount lustre-MDT0000 complete [ 881.633085] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 881.639905] LustreError: Skipped 1 previous similar message [ 899.040186] Lustre: 3631:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787692984/real 1787692984] req@ffff8bc1c768e680 x1874531101472512/t0(0) o400->MGC192.168.203.123@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787693000 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 899.086921] LustreError: MGC192.168.203.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 900.172059] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 909.298199] LustreError: 3628:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8bc2d7729f80 x1874531101481472/t0(0) o250->MGC192.168.203.123@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 909.758507] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 909.769243] Lustre: Skipped 3 previous similar messages [ 915.019659] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 915.053274] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:123 to 0x2c0000401:225) [ 915.057868] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 925.558488] Lustre: DEBUG MARKER: oleg323-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 927.149533] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 930.297692] Lustre: *** cfs_fail_loc=194, val=0*** [ 930.307480] Lustre: 6500:0:(osd_handler.c:873:osd_check_lma()) lustre-MDT0000: unsupported incompat LMA feature(s) 0x0 for fid = [0x200000402:0x3a6:0x0], ino = 512169 [ 937.419941] Lustre: DEBUG MARKER: == sanity test 17p: symlink overwrite directory error message ========================================================== 17:23:57 (1787693037) [ 945.255257] Lustre: DEBUG MARKER: == sanity test 17q: set large xattr on fast symlink ====== 17:24:05 (1787693045) [ 949.161961] sysctl (33300): drop_caches: 3 [ 955.785870] Lustre: DEBUG MARKER: == sanity test 18: touch .../f ; ls ... ======================================================================================== 17:24:15 (1787693055) [ 962.322523] Lustre: DEBUG MARKER: == sanity test 19a: touch .../f19 ; ls -l ... ; rm .../f19 ===================================================================== 17:24:22 (1787693062) [ 970.040801] Lustre: DEBUG MARKER: == sanity test 19b: ls -l .../f19 (should return error) ======================================================================== 17:24:30 (1787693070) [ 976.951422] Lustre: DEBUG MARKER: == sanity test 19c: runas -u 500 -g 500 touch .../f19 (should return error) ============================================================ 17:24:36 (1787693076) [ 983.105990] Lustre: DEBUG MARKER: == sanity test 19d: cat .../f19 (should return error) ======================================================================== 17:24:43 (1787693083) [ 988.634721] Lustre: DEBUG MARKER: == sanity test 20: touch .../f ; ls -l ... =============== 17:24:48 (1787693088) [ 996.516787] Lustre: DEBUG MARKER: == sanity test 21: write to dangling link ================ 17:24:56 (1787693096) [ 1004.198595] Lustre: DEBUG MARKER: == sanity test 22: unpack tar archive as non-root user === 17:25:03 (1787693103) [ 1011.544182] Lustre: DEBUG MARKER: == sanity test 23a: O_CREAT|O_EXCL in subdir ============= 17:25:11 (1787693111) [ 1017.373620] Lustre: DEBUG MARKER: == sanity test 23b: O_APPEND check ======================= 17:25:17 (1787693117) [ 1024.820155] Lustre: DEBUG MARKER: == sanity test 23c: O_APPEND size checks for tiny writes ========================================================== 17:25:24 (1787693124) [ 1038.075977] Lustre: DEBUG MARKER: == sanity test 23d: file offset is correct after appending writes ========================================================== 17:25:38 (1787693138) [ 1044.286867] Lustre: DEBUG MARKER: == sanity test 23e: tiny write updates the size seen by cached statx ========================================================== 17:25:44 (1787693144) [ 1050.662900] Lustre: DEBUG MARKER: == sanity test 24a: rename file to non-existent target === 17:25:50 (1787693150) [ 1057.700285] Lustre: DEBUG MARKER: == sanity test 24b: rename file to existing target ======= 17:25:57 (1787693157) [ 1065.142866] Lustre: DEBUG MARKER: == sanity test 24c: rename directory to non-existent target ========================================================== 17:26:04 (1787693164) [ 1072.289406] Lustre: DEBUG MARKER: == sanity test 24d: rename directory to existing target == 17:26:12 (1787693172) [ 1078.855417] Lustre: DEBUG MARKER: == sanity test 24e: touch .../R5a/f; rename .../R5a/f .../R5b/g ================================================================ 17:26:18 (1787693178) [ 1084.882449] Lustre: DEBUG MARKER: == sanity test 24f: touch .../R6a/f R6b/g; mv .../R6a/f .../R6b/g ============================================================== 17:26:24 (1787693184) [ 1092.105833] Lustre: DEBUG MARKER: == sanity test 24g: mkdir .../R7a/d; .../R7b/d; mv .../R7a/d .../R7b/e ================================================================ 17:26:31 (1787693191) [ 1098.990738] Lustre: DEBUG MARKER: == sanity test 24h: mkdir .../R8a/d; .../R8a/e; .../R8b/d; .../R8b/e; rename .../R8a/d .../R8b/e ========================================================== 17:26:39 (1787693199) [ 1106.923378] Lustre: DEBUG MARKER: == sanity test 24i: rename file to dir error: touch f ; mkdir a ; rename f a ========================================================== 17:26:46 (1787693206) [ 1113.108640] Lustre: DEBUG MARKER: == sanity test 24j: source does not exist ====================================================================================== 17:26:53 (1787693213) [ 1118.644951] Lustre: DEBUG MARKER: == sanity test 24k: touch .../R11a/f; mv .../R11a/f .../R11a/d ================================================================= 17:26:58 (1787693218) [ 1127.424462] Lustre: DEBUG MARKER: == sanity test 24l: Renaming a file to itself ================================================================================== 17:27:06 (1787693226) [ 1135.100349] Lustre: DEBUG MARKER: == sanity test 24m: Renaming a file to a hard link to itself =================================================================== 17:27:14 (1787693234) [ 1142.101312] Lustre: DEBUG MARKER: == sanity test 24n: Statting the old file after renaming (Posix rename 2) ========================================================== 17:27:22 (1787693242) [ 1148.589970] Lustre: DEBUG MARKER: == sanity test 24o: rename of files during htree split === 17:27:28 (1787693248) [ 1835.985719] Lustre: DEBUG MARKER: == sanity test 24p: mkdir .../R12a; .../R12b; rename .../R12a .../R12b ========================================================== 17:38:56 (1787693936) [ 1842.294751] Lustre: DEBUG MARKER: == sanity test 24q: mkdir .../R13a; .../R13b; open R13b rename R13a R13b ============================================================= 17:39:02 (1787693942) [ 1849.108049] Lustre: DEBUG MARKER: == sanity test 24r: mkdir .../R14a/b; rename .../R14a .../R14a/b =============================================================== 17:39:08 (1787693948) [ 1855.639657] Lustre: DEBUG MARKER: == sanity test 24s: mkdir .../R15a/b/c; rename .../R15a .../R15a/b/c =========================================================== 17:39:15 (1787693955) [ 1861.921900] Lustre: DEBUG MARKER: == sanity test 24t: mkdir .../R16a/b/c; rename .../R16a/b/c .../R16a =========================================================== 17:39:21 (1787693961) [ 1868.649910] Lustre: DEBUG MARKER: == sanity test 24u: create stripe file =================== 17:39:28 (1787693968) [ 1876.779445] Lustre: DEBUG MARKER: == sanity test 24v: list large directory (test hash collision, b=17560) ========================================================== 17:39:36 (1787693976) [ 3660.575413] Lustre: DEBUG MARKER: == sanity test 24w: Reading a file larger than 4Gb ======= 18:09:20 (1787695760) [ 3667.731201] Lustre: DEBUG MARKER: == sanity test 24x: cross MDT rename/link ================ 18:09:27 (1787695767) [ 3676.076074] Lustre: DEBUG MARKER: == sanity test 24y: rename/link on the same dir should succeed ========================================================== 18:09:35 (1787695775) [ 3684.796777] Lustre: DEBUG MARKER: == sanity test 24z: cross-MDT rename is done as cp ======= 18:09:44 (1787695784) [ 3696.099633] Lustre: DEBUG MARKER: == sanity test 24A: readdir() returns correct number of entries. ========================================================== 18:09:55 (1787695795) [ 3799.709968] Lustre: DEBUG MARKER: == sanity test 24B: readdir for striped dir return correct number of entries ========================================================== 18:11:39 (1787695899) [ 3807.330682] Lustre: DEBUG MARKER: == sanity test 24C: check .. in striped dir ============== 18:11:47 (1787695907) [ 3815.054464] Lustre: DEBUG MARKER: == sanity test 24E: cross MDT rename/link ================ 18:11:55 (1787695915) [ 3816.422804] Lustre: DEBUG MARKER: SKIP: sanity test_24E needs >= 4 MDTs [ 3818.254267] Lustre: DEBUG MARKER: == sanity test 24F: hash order vs readdir (LU-11330) ===== 18:11:58 (1787695918) [ 3871.621863] Lustre: DEBUG MARKER: == sanity test 24G: migrate symlink in rename ============ 18:12:51 (1787695971) [ 3878.651880] Lustre: DEBUG MARKER: == sanity test 24H: repeat FLD_QUERY rpc ================= 18:12:58 (1787695978) [ 3886.234587] Lustre: DEBUG MARKER: == sanity test 24I: large striped mkdir with limit check ========================================================== 18:13:06 (1787695986) [ 3893.322288] Lustre: DEBUG MARKER: == sanity test 25a: create file in symlinked directory ========================================================================= 18:13:13 (1787695993) [ 3900.456768] Lustre: DEBUG MARKER: == sanity test 25b: lookup file in symlinked directory ========================================================================= 18:13:20 (1787696000) [ 3907.218337] Lustre: DEBUG MARKER: == sanity test 26a: multiple component symlink ================================================================================= 18:13:26 (1787696006) [ 3915.730441] Lustre: DEBUG MARKER: == sanity test 26b: multiple component symlink at end of lookup ================================================================ 18:13:35 (1787696015) [ 3923.648828] Lustre: DEBUG MARKER: == sanity test 26c: chain of symlinks ==================== 18:13:43 (1787696023) [ 3930.209398] Lustre: DEBUG MARKER: == sanity test 26d: create multiple component recursive symlink ========================================================== 18:13:50 (1787696030) [ 3937.906517] Lustre: DEBUG MARKER: == sanity test 26e: unlink multiple component recursive symlink ========================================================== 18:13:57 (1787696037) [ 3945.101393] Lustre: DEBUG MARKER: == sanity test 26f: rm -r of a directory which has recursive symlink ========================================================== 18:14:04 (1787696044) [ 3955.194911] Lustre: DEBUG MARKER: == sanity test 27a: one stripe file ====================== 18:14:14 (1787696054) [ 3961.571988] Lustre: DEBUG MARKER: == sanity test 27b: create and write to two stripe file == 18:14:21 (1787696061) [ 3968.702326] Lustre: DEBUG MARKER: == sanity test 27ca: one stripe on specified OST ========= 18:14:28 (1787696068) [ 3976.039051] Lustre: DEBUG MARKER: == sanity test 27cb: two stripes on specified OSTs ======= 18:14:36 (1787696076) [ 3984.825701] Lustre: DEBUG MARKER: == sanity test 27cc: two stripes on the same OST ========= 18:14:44 (1787696084) [ 3992.461814] Lustre: DEBUG MARKER: == sanity test 27cd: four stripes on two OSTs ============ 18:14:52 (1787696092) [ 4000.115266] Lustre: DEBUG MARKER: == sanity test 27ce: more stripes than OSTs with -o ====== 18:15:00 (1787696100) [ 4008.589487] Lustre: DEBUG MARKER: == sanity test 27cf: 'setstripe -o' on inactive OSTs should return error ========================================================== 18:15:08 (1787696108) [ 4009.770074] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4012.714512] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4012.735674] Lustre: Skipped 7 previous similar messages [ 4012.747757] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4012.765425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.784318] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4012.797295] Lustre: Skipped 6 previous similar messages [ 4012.798476] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.408191] Lustre: DEBUG MARKER: == sanity test 27cg: 1000 shouldn't cause too many credits ========================================================== 18:15:20 (1787696120) [ 4025.767630] Lustre: 16756:0:(osd_handler.c:2191:osd_trans_start()) lustre-MDT0000: credits 30549 > trans_max 3200 [ 4025.780367] Lustre: 16756:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 4/16/0, destroy: 1/4/0 [ 4025.791635] Lustre: 16756:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 2003/2003/0, xattr_set: 3004/28292/0 [ 4025.802889] Lustre: 16756:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 20/142/0, punch: 0/0/0, quota 0/0/0 [ 4025.820064] Lustre: 16756:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 5/84/0, delete: 2/5/0 [ 4025.830987] Lustre: 16756:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 2/2/0 [ 4025.841962] CPU: 2 PID: 16756 Comm: mdt00_004 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 [ 4025.850493] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4025.857974] Call Trace: [ 4025.860375] ? dump_stack+0xbb/0x10e [ 4025.863723] ? osd_trans_start+0x8db/0x9d0 [osd_ldiskfs] [ 4025.867598] ? dt_trans_start+0x1c/0x70 [obdclass] [ 4025.872635] ? top_trans_start+0x4fd/0xd00 [ptlrpc] [ 4025.877015] ? lod_declare_ref_add+0x1a/0x30 [lod] [ 4025.881681] ? dt_declare_ref_add+0x6c/0x200 [obdclass] [ 4025.885694] ? lod_trans_start+0x109/0x4c0 [lod] [ 4025.889365] ? mdd_declare_links_add+0x60/0x90 [mdd] [ 4025.891260] ? mdd_trans_start+0x18/0x30 [mdd] [ 4025.896985] ? mdd_unlink+0x7a4/0x13c0 [mdd] [ 4025.900796] ? mdt_reint_unlink+0x14aa/0x1a30 [mdt] [ 4025.904351] ? mdt_reint_rec+0x139/0x2b0 [mdt] [ 4025.906751] ? mdt_reint_internal+0x693/0xdc0 [mdt] [ 4025.909956] ? mdt_reint+0x163/0x190 [mdt] [ 4025.912211] ? tgt_handle_request0+0x137/0xaf0 [ptlrpc] [ 4025.916565] ? tgt_request_handle+0x575/0x1f70 [ptlrpc] [ 4025.922752] ? ptlrpc_server_handle_request+0x443/0x13b0 [ptlrpc] [ 4025.928573] ? lprocfs_counter_add+0x15b/0x210 [obdclass] [ 4025.938377] ? ptlrpc_main+0xce8/0x1400 [ptlrpc] [ 4025.943687] ? ptlrpc_wait_event+0x690/0x690 [ptlrpc] [ 4025.949600] ? kthread+0x1d1/0x200 [ 4025.952487] ? set_kthread_struct+0x70/0x70 [ 4025.955522] ? ret_from_fork+0x1f/0x30 [ 4035.811269] Lustre: DEBUG MARKER: == sanity test 27d: create file with default settings ==== 18:15:35 (1787696135) [ 4042.602414] Lustre: DEBUG MARKER: == sanity test 27e: setstripe existing file (should return error) ========================================================== 18:15:42 (1787696142) [ 4049.327380] Lustre: DEBUG MARKER: == sanity test 27f: setstripe with bad stripe size (should return error) ========================================================== 18:15:49 (1787696149) [ 4056.449827] Lustre: DEBUG MARKER: == sanity test 27g: /home/green/git/lustre-release/lustre/utils/lfs getstripe with no objects ========================================================== 18:15:56 (1787696156) [ 4064.390936] Lustre: DEBUG MARKER: == sanity test 27ga: /home/green/git/lustre-release/lustre/utils/lfs getstripe with missing file (should return error) ========================================================== 18:16:03 (1787696163) [ 4074.607314] Lustre: DEBUG MARKER: == sanity test 27i: /home/green/git/lustre-release/lustre/utils/lfs getstripe with some objects ========================================================== 18:16:13 (1787696173) [ 4084.982758] Lustre: DEBUG MARKER: == sanity test 27j: setstripe with bad stripe offset (should return error) ========================================================== 18:16:24 (1787696184) [ 4094.615453] Lustre: DEBUG MARKER: == sanity test 27k: limit i_blksize for broken user apps ========================================================== 18:16:34 (1787696194) [ 4103.593262] Lustre: DEBUG MARKER: == sanity test 27l: check setstripe permissions (should return error) ========================================================== 18:16:43 (1787696203) [ 4111.138224] Lustre: DEBUG MARKER: SKIP: sanity test_27m skipping SLOW test 27m [ 4113.418705] Lustre: DEBUG MARKER: == sanity test 27n: create file with some full OSTs ====== 18:16:52 (1787696212) [ 4128.634118] Lustre: *** cfs_fail_loc=215, val=0*** [ 4130.802305] Lustre: *** cfs_fail_loc=215, val=0*** [ 4130.810027] Lustre: Skipped 1 previous similar message [ 4135.913697] Lustre: *** cfs_fail_loc=215, val=0*** [ 4162.819401] Lustre: DEBUG MARKER: == sanity test 27o: create file with all full OSTs (should error) ========================================================== 18:17:42 (1787696262) [ 4175.330788] Lustre: *** cfs_fail_loc=215, val=1*** [ 4176.874613] LustreError: 10078:0:(lod_qos.c:1827:lod_ost_alloc_specific()) can't lstripe objid [0x200000402:0xd23d:0x0]: have 0 want 1 [ 4180.451657] Lustre: *** cfs_fail_loc=215, val=1*** [ 4180.453163] Lustre: Skipped 3 previous similar messages [ 4183.255904] LustreError: 16756:0:(lod_qos.c:1827:lod_ost_alloc_specific()) can't lstripe objid [0x200000402:0xd241:0x0]: have 0 want 1 [ 4185.572643] Lustre: *** cfs_fail_loc=215, val=1*** [ 4185.579444] Lustre: Skipped 3 previous similar messages [ 4195.808153] Lustre: *** cfs_fail_loc=215, val=1*** [ 4195.815846] Lustre: Skipped 7 previous similar messages [ 4211.410537] Lustre: DEBUG MARKER: == sanity test 27oo: don't let few threads to reserve too many objects ========================================================== 18:18:31 (1787696311) [ 4235.611443] Lustre: Failing over lustre-OST0000 [ 4235.828144] Lustre: server umount lustre-OST0000 complete [ 4236.791779] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4236.803058] Lustre: Skipped 1 previous similar message [ 4236.827989] LustreError: 25108:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4236.847549] LustreError: 25108:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 38 previous similar messages [ 4251.415806] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4251.729796] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4251.751605] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4251.761712] Lustre: Skipped 1 previous similar message [ 4253.210445] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4253.215526] Lustre: Skipped 1 previous similar message [ 4253.603848] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4253.604195] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 4253.610065] Lustre: Skipped 1 previous similar message [ 4253.623886] Lustre: Skipped 1 previous similar message [ 4257.999422] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 4263.141481] Lustre: DEBUG MARKER: == sanity test 27p: append to a truncated file with some full OSTs ========================================================== 18:19:23 (1787696363) [ 4279.264170] Lustre: *** cfs_fail_loc=215, val=0*** [ 4279.274455] Lustre: Skipped 4 previous similar messages [ 4311.218913] Lustre: DEBUG MARKER: == sanity test 27q: append to truncated file with all OSTs full (should error) ========================================================== 18:20:10 (1787696410) [ 4323.812524] Lustre: *** cfs_fail_loc=215, val=1*** [ 4323.814361] Lustre: Skipped 3 previous similar messages [ 4329.618366] LustreError: 10078:0:(lod_qos.c:1827:lod_ost_alloc_specific()) can't lstripe objid [0x200000402:0xd27c:0x0]: have 0 want 1 [ 4357.111339] Lustre: DEBUG MARKER: == sanity test 27r: stripe file with some full OSTs (shouldn't LBUG) =========================================================== 18:20:57 (1787696457) [ 4394.697694] Lustre: DEBUG MARKER: == sanity test 27s: lsm_xfersize overflow (should error) (bug 10725) ========================================================== 18:21:34 (1787696494) [ 4401.775771] Lustre: DEBUG MARKER: == sanity test 27t: check that utils parse path correctly ========================================================== 18:21:41 (1787696501) [ 4408.914743] Lustre: DEBUG MARKER: == sanity test 27u: skip object creation on OSC w/o objects ========================================================== 18:21:49 (1787696509) [ 4412.868393] Lustre: *** cfs_fail_loc=139, val=0*** [ 4413.371572] Lustre: *** cfs_fail_loc=139, val=0*** [ 4413.375337] Lustre: Skipped 61 previous similar messages [ 4414.384504] Lustre: *** cfs_fail_loc=139, val=0*** [ 4414.387544] Lustre: Skipped 177 previous similar messages [ 4416.386497] Lustre: *** cfs_fail_loc=139, val=0*** [ 4416.392188] Lustre: Skipped 383 previous similar messages [ 4420.394072] Lustre: *** cfs_fail_loc=139, val=0*** [ 4420.401143] Lustre: Skipped 637 previous similar messages [ 4472.359213] Lustre: DEBUG MARKER: == sanity test 27v: skip object creation on slow OST ===== 18:22:52 (1787696572) [ 4477.414257] Lustre: *** cfs_fail_loc=215, val=1*** [ 4477.416152] Lustre: Skipped 19 previous similar messages [ 4483.258908] LustreError: 10078:0:(lod_qos.c:1827:lod_ost_alloc_specific()) can't lstripe objid [0x200000402:0xd499:0x0]: have 0 want 1 [ 4529.703582] Lustre: DEBUG MARKER: == sanity test 27w: check /home/green/git/lustre-release/lustre/utils/lfs setstripe -S and getstrip -d options ========================================================== 18:23:49 (1787696629) [ 4537.186296] Lustre: DEBUG MARKER: == sanity test 27wa: check /home/green/git/lustre-release/lustre/utils/lfs setstripe -c -i options ========================================================== 18:23:57 (1787696637) [ 4545.647759] Lustre: DEBUG MARKER: == sanity test 27x: create files while OST0 is degraded == 18:24:05 (1787696645) [ 4565.794961] Lustre: DEBUG MARKER: == sanity test 27y: create files while OST0 is degraded and the rest inactive ========================================================== 18:24:25 (1787696665) [ 4569.887492] Lustre: setting import lustre-OST0001_UUID INACTIVE by administrator request [ 4569.891959] Lustre: Skipped 1 previous similar message [ 4587.527767] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4587.545470] Lustre: Skipped 1 previous similar message [ 4587.560404] Lustre: lustre-OST0001: Client lustre-MDT0001-mdtlov_UUID (at 0@lo) reconnecting [ 4587.571838] Lustre: Skipped 1 previous similar message [ 4587.577314] LustreError: lustre-OST0001-osc-MDT0001: This client was evicted by lustre-OST0001; in progress operations using this service will fail. [ 4587.599860] Lustre: lustre-OST0001-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 4587.614194] Lustre: Skipped 1 previous similar message [ 4588.589982] Lustre: lustre-OST0001: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4588.596482] LustreError: lustre-OST0001-osc-MDT0000: This client was evicted by lustre-OST0001; in progress operations using this service will fail. [ 4607.842157] Lustre: DEBUG MARKER: == sanity test 27z: check SEQ/OID on the MDT and OST filesystems ========================================================== 18:25:07 (1787696707) [ 4615.713067] Lustre: DEBUG MARKER: check file /mnt/lustre/d27z.sanity/f27z.sanity-1 [ 4617.450700] Lustre: DEBUG MARKER: FID seq 0x200000402, oid 0xd4c3 ver 0x0 [ 4618.685828] Lustre: DEBUG MARKER: LOV seq 0x200000402, oid 0xd4c3, count: 1 [ 4620.346239] Lustre: DEBUG MARKER: want: stripe:0 ost:0 oid:920/0x398 seq:0x280000401 [ 4623.391812] Lustre: DEBUG MARKER: check file /mnt/lustre/d27z.sanity/f27z.sanity-2 [ 4625.167637] Lustre: DEBUG MARKER: FID seq 0x200000402, oid 0xd4c5 ver 0x0 [ 4627.081294] Lustre: DEBUG MARKER: LOV seq 0x200000402, oid 0xd4c5, count: 2 [ 4628.590456] Lustre: DEBUG MARKER: want: stripe:0 ost:1 oid:1345/0x541 seq:0x2c0000401 [ 4631.608082] Lustre: DEBUG MARKER: want: stripe:1 ost:0 oid:921/0x399 seq:0x280000401 [ 4639.553382] Lustre: DEBUG MARKER: == sanity test 27A: check filesystem-wide default LOV EA values ========================================================== 18:25:39 (1787696739) [ 4647.279645] Lustre: DEBUG MARKER: == sanity test 27B: call setstripe on open unlinked file/rename victim ========================================================== 18:25:47 (1787696747) [ 4653.497906] Lustre: DEBUG MARKER: == sanity test 27Ca: check full striping across all OSTs ========================================================== 18:25:53 (1787696753) [ 4660.609325] Lustre: DEBUG MARKER: == sanity test 27Cb: more stripes than OSTs with -C ====== 18:26:00 (1787696760) [ 4669.437642] Lustre: DEBUG MARKER: == sanity test 27Cc: fewer stripes than OSTs does not set overstriping ========================================================== 18:26:08 (1787696768) [ 4678.110820] Lustre: DEBUG MARKER: == sanity test 27Cd: test maximum stripe count =========== 18:26:18 (1787696778) [ 4686.122327] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x2c0000401 to 0x2c0000402 [ 4686.936900] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x280000401 to 0x280000402 [ 4687.067060] Lustre: lustre-OST0001-osc-MDT0001: update sequence from 0x2c0000400 to 0x2c0000403 [ 4687.365610] Lustre: lustre-OST0000-osc-MDT0001: update sequence from 0x280000400 to 0x280000403 [ 4707.621983] Lustre: 16756:0:(osd_handler.c:2191:osd_trans_start()) lustre-MDT0000: credits 60549 > trans_max 3200 [ 4707.636342] Lustre: 16756:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 4/16/0, destroy: 1/4/0 [ 4707.642840] Lustre: 16756:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 4003/4003/0, xattr_set: 6004/56292/0 [ 4707.655113] Lustre: 16756:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 20/142/0, punch: 0/0/0, quota 0/0/0 [ 4707.672478] Lustre: 16756:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 5/84/0, delete: 2/5/0 [ 4707.682314] Lustre: 16756:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 2/2/0 [ 4707.689306] CPU: 3 PID: 16756 Comm: mdt00_004 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 [ 4707.700884] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4707.708472] Call Trace: [ 4707.710280] ? dump_stack+0xbb/0x10e [ 4707.714330] ? osd_trans_start+0x8db/0x9d0 [osd_ldiskfs] [ 4707.717150] ? dt_trans_start+0x1c/0x70 [obdclass] [ 4707.722535] ? top_trans_start+0x4fd/0xd00 [ptlrpc] [ 4707.730977] ? lod_declare_ref_add+0x1a/0x30 [lod] [ 4707.737751] ? dt_declare_ref_add+0x6c/0x200 [obdclass] [ 4707.742050] ? lod_trans_start+0x109/0x4c0 [lod] [ 4707.750740] ? mdd_declare_links_add+0x60/0x90 [mdd] [ 4707.758918] ? mdd_trans_start+0x18/0x30 [mdd] [ 4707.762995] ? mdd_unlink+0x7a4/0x13c0 [mdd] [ 4707.771386] ? mdt_reint_unlink+0x14aa/0x1a30 [mdt] [ 4707.779894] ? mdt_reint_rec+0x139/0x2b0 [mdt] [ 4707.783841] ? mdt_reint_internal+0x693/0xdc0 [mdt] [ 4707.786892] ? mdt_reint+0x163/0x190 [mdt] [ 4707.789055] ? tgt_handle_request0+0x137/0xaf0 [ptlrpc] [ 4707.792491] ? tgt_request_handle+0x575/0x1f70 [ptlrpc] [ 4707.799397] ? ptlrpc_server_handle_request+0x443/0x13b0 [ptlrpc] [ 4707.803558] ? lprocfs_counter_add+0x15b/0x210 [obdclass] [ 4707.807657] ? ptlrpc_main+0xce8/0x1400 [ptlrpc] [ 4707.811744] ? ptlrpc_wait_event+0x690/0x690 [ptlrpc] [ 4707.813896] ? kthread+0x1d1/0x200 [ 4707.816295] ? set_kthread_struct+0x70/0x70 [ 4707.820606] ? ret_from_fork+0x1f/0x30 [ 4707.823154] Lustre: 16756:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 0: before 3202 < left 4003, rollback = 0 [ 4718.440144] Lustre: DEBUG MARKER: == sanity test 27Ce: test pool with overstriping ========= 18:26:57 (1787696817) [ 4747.324457] Lustre: DEBUG MARKER: == sanity test 27Cf: test default inheritance with overstriping ========================================================== 18:27:26 (1787696846) [ 4756.554476] Lustre: DEBUG MARKER: == sanity test 27Cg: test setstripe with wrong OST idx === 18:27:36 (1787696856) [ 4764.991585] Lustre: DEBUG MARKER: == sanity test 27Ci: add an overstriping component ======= 18:27:44 (1787696864) [ 4772.671991] Lustre: 6500:0:(osd_handler.c:2191:osd_trans_start()) lustre-MDT0000: credits 3579 > trans_max 3200 [ 4772.679991] Lustre: 6500:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 4/16/0, destroy: 1/4/0 [ 4772.695428] Lustre: 6500:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 5 previous similar messages [ 4772.713231] Lustre: 6500:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 205/205/0, xattr_set: 307/3120/0 [ 4772.722139] Lustre: 6500:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 5 previous similar messages [ 4772.731301] Lustre: 6500:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 20/142/0, punch: 0/0/0, quota 0/0/0 [ 4772.737499] Lustre: 6500:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 5 previous similar messages [ 4772.746555] Lustre: 6500:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 5/84/0, delete: 2/5/0 [ 4772.763610] Lustre: 6500:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 5 previous similar messages [ 4772.771320] Lustre: 6500:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 2/2/0 [ 4772.778829] Lustre: 6500:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 5 previous similar messages [ 4772.793487] CPU: 0 PID: 6500 Comm: mdt00_000 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #2 [ 4772.801086] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 4772.808519] Call Trace: [ 4772.809854] ? dump_stack+0xbb/0x10e [ 4772.811947] ? osd_trans_start+0x8db/0x9d0 [osd_ldiskfs] [ 4772.817620] ? dt_trans_start+0x1c/0x70 [obdclass] [ 4772.822582] ? top_trans_start+0x4fd/0xd00 [ptlrpc] [ 4772.825301] ? lod_declare_ref_add+0x1a/0x30 [lod] [ 4772.827386] ? dt_declare_ref_add+0x6c/0x200 [obdclass] [ 4772.830253] ? lod_trans_start+0x109/0x4c0 [lod] [ 4772.832532] ? mdd_declare_links_add+0x60/0x90 [mdd] [ 4772.835315] ? mdd_trans_start+0x18/0x30 [mdd] [ 4772.837543] ? mdd_unlink+0x7a4/0x13c0 [mdd] [ 4772.839689] ? mdt_reint_unlink+0x14aa/0x1a30 [mdt] [ 4772.842158] ? mdt_reint_rec+0x139/0x2b0 [mdt] [ 4772.847203] ? mdt_reint_internal+0x693/0xdc0 [mdt] [ 4772.851329] ? mdt_reint+0x163/0x190 [mdt] [ 4772.856778] ? tgt_handle_request0+0x137/0xaf0 [ptlrpc] [ 4772.861870] ? tgt_request_handle+0x575/0x1f70 [ptlrpc] [ 4772.867946] ? ptlrpc_server_handle_request+0x443/0x13b0 [ptlrpc] [ 4772.876302] ? lprocfs_counter_add+0x15b/0x210 [obdclass] [ 4772.882595] ? ptlrpc_main+0xce8/0x1400 [ptlrpc] [ 4772.886334] ? ptlrpc_wait_event+0x690/0x690 [ptlrpc] [ 4772.890452] ? kthread+0x1d1/0x200 [ 4772.896392] ? set_kthread_struct+0x70/0x70 [ 4772.900455] ? ret_from_fork+0x1f/0x30 [ 4775.472298] Lustre: DEBUG MARKER: == sanity test 27Cj: overstriping with -C for max values in multiple of targets ========================================================== 18:27:55 (1787696875) [ 4782.369060] Lustre: DEBUG MARKER: == sanity test 27D: validate llapi_layout API ============ 18:28:02 (1787696882) [ 4815.417920] Lustre: DEBUG MARKER: == sanity test 27E: check that default extended attribute size properly increases ========================================================== 18:28:35 (1787696915) [ 4822.661178] Lustre: DEBUG MARKER: == sanity test 27F: Client resend delayed layout creation with non-zero size ========================================================== 18:28:41 (1787696921) [ 4824.898419] Lustre: Failing over lustre-OST0000 [ 4825.016764] Lustre: server umount lustre-OST0000 complete [ 4825.059970] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4825.084861] Lustre: Skipped 1 previous similar message [ 4825.089834] LustreError: 47749:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4825.104721] LustreError: 47749:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 4828.589531] Lustre: Failing over lustre-OST0001 [ 4828.640621] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4828.657804] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4828.855597] Lustre: server umount lustre-OST0001 complete [ 4837.249774] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4837.690079] Lustre: lustre-OST0000: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4837.706197] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 4844.319367] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 4846.048939] Lustre: 3632:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787696931/real 1787696931] req@ffff8bc1c7cd1f80 x1874531104846848/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 1 dl 1787696947 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 [ 4846.067329] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 4847.073555] Lustre: 3629:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787696932/real 1787696932] req@ffff8bc2c9f33b80 x1874531104847872/t0(0) o400->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787696948 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 4847.223837] Lustre: lustre-OST0000: Recovery over after 0:01, of 3 clients 3 recovered and 0 were evicted. [ 4847.232178] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 4847.262749] Lustre: Skipped 1 previous similar message [ 4852.197750] Lustre: 3629:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787696937/real 1787696937] req@ffff8bc2c60ca680 x1874531104849664/t0(0) o400->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787696953 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 4852.242418] Lustre: 3629:0:(client.c:2503:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 4854.586433] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4854.960428] Lustre: lustre-OST0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 4854.992752] Lustre: lustre-OST0001: in recovery but waiting for the first client to connect [ 4862.066364] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug all all [ 4871.276716] Lustre: DEBUG MARKER: == sanity test 27G: Clear OST pool from stripe =========== 18:29:31 (1787696971) [ 4897.471279] Lustre: DEBUG MARKER: == sanity test 27H: Set specific OSTs stripe ============= 18:29:57 (1787696997) [ 4898.956793] Lustre: DEBUG MARKER: SKIP: sanity test_27H needs >= 3 OSTs [ 4900.637321] Lustre: DEBUG MARKER: == sanity test 27I: check that root dir striping does not break parent dir one ========================================================== 18:30:00 (1787697000) [ 4924.304441] Lustre: DEBUG MARKER: == sanity test 27Ia: check that root dir pool is dropped with conflict parent dir settings ========================================================== 18:30:24 (1787697024) [ 4958.768984] Lustre: DEBUG MARKER: == sanity test 27J: basic ops on file with foreign LOV === 18:30:58 (1787697058) [ 4966.281869] Lustre: DEBUG MARKER: == sanity test 27K: basic ops on dir with foreign LMV ==== 18:31:06 (1787697066) [ 4974.452149] Lustre: DEBUG MARKER: == sanity test 27Ke: test enable_foreign_dir and enable_foreign_dir_gid ========================================================== 18:31:14 (1787697074) [ 5001.501955] Lustre: DEBUG MARKER: == sanity test 27L: lfs pool_list gives correct pool name ========================================================== 18:31:41 (1787697101) [ 5020.718644] Lustre: DEBUG MARKER: == sanity test 27M: test O_APPEND striping =============== 18:32:00 (1787697120) [ 5078.724929] Lustre: DEBUG MARKER: == sanity test 27N: lctl pool_list on separate MGS gives correct pool name ========================================================== 18:32:58 (1787697178) [ 5080.141915] Lustre: DEBUG MARKER: SKIP: sanity test_27N needs separate MGS/MDT [ 5081.761268] Lustre: DEBUG MARKER: == sanity test 27O: basic ops on foreign file of symlink type ========================================================== 18:33:01 (1787697181) [ 5089.488061] Lustre: DEBUG MARKER: == sanity test 27P: basic ops on foreign dir of foreign_symlink type ========================================================== 18:33:09 (1787697189) [ 5098.531353] Lustre: DEBUG MARKER: == sanity test 27Q: llapi_file_get_stripe() works on symlinks ========================================================== 18:33:18 (1787697198) [ 5106.963733] Lustre: DEBUG MARKER: == sanity test 27R: test max_stripecount limitation when stripe count is set to -1 ========================================================== 18:33:26 (1787697206) [ 5118.409643] Lustre: DEBUG MARKER: == sanity test 27T: no eio on close on partial write due to enosp ========================================================== 18:33:38 (1787697218) [ 5126.346846] Lustre: *** cfs_fail_loc=255, val=0*** [ 5133.698331] Lustre: DEBUG MARKER: == sanity test 27U: append pool and stripe count work with composite default layout ========================================================== 18:33:53 (1787697233) [ 5191.714507] Lustre: DEBUG MARKER: == sanity test 27V: creating widely striped file races with deactivating OST ========================================================== 18:34:51 (1787697291) [ 5193.643871] Lustre: DEBUG MARKER: SKIP: sanity test_27V needs >= 4 OSTs [ 5195.698548] Lustre: DEBUG MARKER: == sanity test 27W: test enable_setstripe_gid ============ 18:34:55 (1787697295) [ 5207.095323] Lustre: DEBUG MARKER: == sanity test 27X: lfs migrate honors --overstripe-count option ========================================================== 18:35:06 (1787697306) [ 5217.813745] Lustre: DEBUG MARKER: == sanity test 28: create/mknod/mkdir with bad file types ====================================================================== 18:35:17 (1787697317) [ 5226.837414] Lustre: DEBUG MARKER: == sanity test 29: IT_GETATTR regression ====================================================================================== 18:35:26 (1787697326) [ 5231.463620] Lustre: DEBUG MARKER: first d29 [ 5234.680418] Lustre: DEBUG MARKER: second d29 [ 5236.723609] Lustre: DEBUG MARKER: done [ 5244.768742] Lustre: DEBUG MARKER: == sanity test 30a: execute binary from Lustre (execve) ======================================================================== 18:35:44 (1787697344) [ 5252.341437] Lustre: DEBUG MARKER: == sanity test 30b: execute binary from Lustre as non-root ===================================================================== 18:35:52 (1787697352) [ 5259.409240] Lustre: DEBUG MARKER: == sanity test 30c: execute binary from Lustre without read perms ============================================================== 18:35:59 (1787697359) [ 5266.636878] Lustre: DEBUG MARKER: == sanity test 30d: execute binary from Lustre while clear locks ========================================================== 18:36:06 (1787697366) [ 5462.941614] Lustre: DEBUG MARKER: == sanity test 31a: open-unlink file ============================================================================================ 18:39:22 (1787697562) [ 5469.888778] Lustre: DEBUG MARKER: == sanity test 31b: unlink file with multiple links while open ================================================================= 18:39:29 (1787697569) [ 5477.107921] Lustre: DEBUG MARKER: == sanity test 31c: open-unlink file with multiple links ======================================================================= 18:39:37 (1787697577) [ 5484.307431] Lustre: DEBUG MARKER: == sanity test 31d: remove of open directory =================================================================================== 18:39:44 (1787697584) [ 5491.168675] Lustre: DEBUG MARKER: == sanity test 31e: remove of open non-empty directory ========================================================================= 18:39:51 (1787697591) [ 5497.565658] Lustre: DEBUG MARKER: == sanity test 31f: remove of open directory with open-unlink file ============================================================= 18:39:57 (1787697597) [ 5511.269730] Lustre: DEBUG MARKER: == sanity test 31g: cross directory link================== 18:40:11 (1787697611) [ 5517.504534] Lustre: DEBUG MARKER: == sanity test 31h: cross directory link under child========================================================================= 18:40:17 (1787697617) [ 5523.876329] Lustre: DEBUG MARKER: == sanity test 31i: cross directory link under parent========================================================================= 18:40:23 (1787697623) [ 5529.939322] Lustre: DEBUG MARKER: == sanity test 31j: link for directory =================== 18:40:30 (1787697630) [ 5536.313600] Lustre: DEBUG MARKER: == sanity test 31k: link to file: the same, non-existing, dir ========================================================== 18:40:36 (1787697636) [ 5542.375084] Lustre: DEBUG MARKER: == sanity test 31l: link to file: target dir has trailing slash ========================================================== 18:40:42 (1787697642) [ 5548.753905] Lustre: DEBUG MARKER: == sanity test 31m: link to file: the same, non-existing, dir ========================================================== 18:40:48 (1787697648) [ 5556.327913] Lustre: DEBUG MARKER: == sanity test 31n: check link count of unlinked file ==== 18:40:56 (1787697656) [ 5563.214749] Lustre: DEBUG MARKER: == sanity test 31o: duplicate hard links with same filename ========================================================== 18:41:02 (1787697662) [ 5646.832437] Lustre: DEBUG MARKER: == sanity test 31p: remove of open striped directory ===== 18:42:27 (1787697747) [ 5652.561254] Lustre: DEBUG MARKER: == sanity test 31q: create striped directory on specific MDTs ========================================================== 18:42:32 (1787697752) [ 5653.876382] Lustre: DEBUG MARKER: SKIP: sanity test_31q needs >= 3 MDTs [ 5655.511597] Lustre: DEBUG MARKER: == sanity test 31r: open-rename(replace) race ============ 18:42:35 (1787697755) [ 5661.954993] Lustre: DEBUG MARKER: == sanity test 32a: stat d32a/ext2-mountpoint/.. =============================================================================== 18:42:42 (1787697762) [ 5669.023412] Lustre: DEBUG MARKER: == sanity test 32b: open d32b/ext2-mountpoint/.. =============================================================================== 18:42:48 (1787697768) [ 5678.453985] Lustre: DEBUG MARKER: == sanity test 32c: stat d32c/ext2-mountpoint/../d2/test_dir =================================================================== 18:42:58 (1787697778) [ 5686.118207] Lustre: DEBUG MARKER: == sanity test 32d: open d32d/ext2-mountpoint/../d2/test_dir ========================================================== 18:43:06 (1787697786) [ 5694.717563] Lustre: DEBUG MARKER: == sanity test 32e: stat d32e/symlink->tmp/symlink->lustre-subdir ========================================================== 18:43:14 (1787697794) [ 5702.583269] Lustre: DEBUG MARKER: == sanity test 32f: open d32f/symlink->tmp/symlink->lustre-subdir ========================================================== 18:43:22 (1787697802) [ 5710.174392] Lustre: DEBUG MARKER: == sanity test 32g: stat d32g/symlink->tmp/symlink->lustre-subdir/2 ========================================================== 18:43:30 (1787697810) [ 5717.427939] Lustre: DEBUG MARKER: == sanity test 32h: open d32h/symlink->tmp/symlink->lustre-subdir/2 ========================================================== 18:43:37 (1787697817) [ 5724.916457] Lustre: DEBUG MARKER: == sanity test 32i: stat d32i/ext2-mountpoint/../test_file ===================================================================== 18:43:44 (1787697824) [ 5732.260905] Lustre: DEBUG MARKER: == sanity test 32j: open d32j/ext2-mountpoint/../test_file ===================================================================== 18:43:51 (1787697831) [ 5739.970530] Lustre: DEBUG MARKER: == sanity test 32k: stat d32k/ext2-mountpoint/../d2/test_file ================================================================== 18:44:00 (1787697840) [ 5747.224376] Lustre: DEBUG MARKER: == sanity test 32l: open d32l/ext2-mountpoint/../d2/test_file ================================================================== 18:44:07 (1787697847) [ 5756.452939] Lustre: DEBUG MARKER: == sanity test 32m: stat d32m/symlink->tmp/symlink->lustre-root ================================================================ 18:44:16 (1787697856) [ 5765.056868] Lustre: DEBUG MARKER: == sanity test 32n: open d32n/symlink->tmp/symlink->lustre-root ================================================================ 18:44:24 (1787697864) [ 5773.071075] Lustre: DEBUG MARKER: == sanity test 32o: stat d32o/symlink->tmp/symlink->lustre-root/ ========================================================== 18:44:32 (1787697872) [ 5780.289891] Lustre: DEBUG MARKER: == sanity test 32p: open d32p/symlink->tmp/symlink->lustre-root/ ========================================================== 18:44:39 (1787697879) [ 5781.835739] Lustre: DEBUG MARKER: 32p_1 [ 5783.362470] Lustre: DEBUG MARKER: 32p_2 [ 5784.879963] Lustre: DEBUG MARKER: 32p_3 [ 5786.637880] Lustre: DEBUG MARKER: 32p_4 [ 5788.627763] Lustre: DEBUG MARKER: 32p_5 [ 5790.227194] Lustre: DEBUG MARKER: 32p_6 [ 5791.893633] Lustre: DEBUG MARKER: 32p_7 [ 5793.413671] Lustre: DEBUG MARKER: 32p_8 [ 5794.710170] Lustre: DEBUG MARKER: 32p_9 [ 5796.806330] Lustre: DEBUG MARKER: 32p_10 [ 5803.805389] Lustre: DEBUG MARKER: == sanity test 32q: stat follows mountpoints in Lustre (should return error) ========================================================== 18:45:03 (1787697903) [ 5810.480564] Lustre: DEBUG MARKER: == sanity test 32r: opendir follows mountpoints in Lustre (should return error) ========================================================== 18:45:10 (1787697910) [ 5817.758560] Lustre: DEBUG MARKER: == sanity test 33aa: write file with mode 444 (should return error) ========================================================== 18:45:17 (1787697917) [ 5819.493805] Lustre: DEBUG MARKER: 33_1 [ 5820.937943] Lustre: DEBUG MARKER: 33_2 [ 5826.961540] Lustre: DEBUG MARKER: == sanity test 33a: test open file(mode=0444) with O_RDWR (should return error) ========================================================== 18:45:27 (1787697927) [ 5833.589274] Lustre: DEBUG MARKER: == sanity test 33b: test open file with malformed flags (No panic) ========================================================== 18:45:33 (1787697933) [ 5840.544472] Lustre: DEBUG MARKER: == sanity test 33c: test write_bytes stats =============== 18:45:40 (1787697940) [ 5848.226950] Lustre: DEBUG MARKER: == sanity test 33d: openfile with 444 modes and malformed flags under remote dir ========================================================== 18:45:48 (1787697948) [ 5855.795114] Lustre: DEBUG MARKER: == sanity test 33e: mkdir and striped directory should have same mode ========================================================== 18:45:55 (1787697955) [ 5864.626630] Lustre: DEBUG MARKER: == sanity test 33f: nonroot user can create, access, and remove a striped directory ========================================================== 18:46:04 (1787697964) [ 5875.628789] Lustre: DEBUG MARKER: == sanity test 33g: nonroot user create already existing root created file ========================================================== 18:46:15 (1787697975) [ 5883.211204] Lustre: DEBUG MARKER: == sanity test 33h: temp file is located on the same MDT as target (crush) ========================================================== 18:46:23 (1787697983) [ 6028.073294] Lustre: DEBUG MARKER: == sanity test 33hh: temp file is located on the same MDT as target (crush2) ========================================================== 18:48:47 (1787698127) [ 6190.859360] Lustre: DEBUG MARKER: == sanity test 33i: striped directory can be accessed when one MDT is down ========================================================== 18:51:30 (1787698290) [ 6245.725117] Lustre: lustre-MDT0001: Client d5c235a1-d50a-4b39-b1bc-0b57968437f6 (at 192.168.203.23@tcp) reconnecting [ 6247.806194] Lustre: DEBUG MARKER: == sanity test 33j: lfs setdirstripe -D -i x,y,x should fail ========================================================== 18:52:27 (1787698347) [ 6254.171138] Lustre: DEBUG MARKER: == sanity test 34a: truncate file that has not been opened ===================================================================== 18:52:34 (1787698354) [ 6260.305754] Lustre: DEBUG MARKER: == sanity test 34b: O_RDONLY opening file doesn't create objects =============================================================== 18:52:40 (1787698360) [ 6266.500981] Lustre: DEBUG MARKER: == sanity test 34c: O_RDWR opening file-with-size works ======================================================================== 18:52:46 (1787698366) [ 6272.913559] Lustre: DEBUG MARKER: == sanity test 34d: write to sparse file ======================================================================================= 18:52:52 (1787698372) [ 6278.938405] Lustre: DEBUG MARKER: == sanity test 34e: create objects, some with size and some without ============================================================ 18:52:58 (1787698378) [ 6285.573467] Lustre: DEBUG MARKER: == sanity test 34f: read from a file with no objects until EOF ================================================================= 18:53:05 (1787698385) [ 6292.069524] Lustre: DEBUG MARKER: == sanity test 34g: truncate long file ========================================================================================= 18:53:11 (1787698391) [ 6298.257290] Lustre: DEBUG MARKER: == sanity test 34h: ftruncate file under grouplock should not block ========================================================== 18:53:18 (1787698398) [ 6307.967670] Lustre: DEBUG MARKER: == sanity test 35a: exec file with mode 444 (should return and not leak) ========================================================== 18:53:27 (1787698407) [ 6315.063646] Lustre: DEBUG MARKER: == sanity test 36a: MDS utime check (mknod, utime) ======= 18:53:35 (1787698415) [ 6322.077915] Lustre: DEBUG MARKER: == sanity test 36b: OST utime check (open, utime) ======== 18:53:41 (1787698421) [ 6329.190911] Lustre: DEBUG MARKER: == sanity test 36c: non-root MDS utime check (mknod, utime) ========================================================== 18:53:49 (1787698429) [ 6337.080835] Lustre: DEBUG MARKER: == sanity test 36d: non-root OST utime check (open, utime) ========================================================== 18:53:56 (1787698436) [ 6343.855860] Lustre: DEBUG MARKER: == sanity test 36e: utime on non-owned file (should return error) ========================================================== 18:54:03 (1787698443) [ 6351.073062] Lustre: DEBUG MARKER: == sanity test 36f: utime on file racing with OST BRW write ==================================================================== 18:54:11 (1787698451) [ 6358.571850] Lustre: DEBUG MARKER: == sanity test 36g: FMD cache expiry =============================================================================== 18:54:18 (1787698458) [ 6411.586677] Lustre: DEBUG MARKER: == sanity test 36h: utime on file racing with OST BRW write ==================================================================== 18:55:11 (1787698511) [ 6420.714391] Lustre: DEBUG MARKER: == sanity test 36i: change mtime on striped directory ==== 18:55:20 (1787698520) [ 6428.952478] Lustre: DEBUG MARKER: == sanity test 38: open a regular file with O_DIRECTORY should return -ENOTDIR ============================================================= 18:55:28 (1787698528) [ 6435.960163] Lustre: DEBUG MARKER: == sanity test 39a: mtime changed on create ============== 18:55:35 (1787698535) [ 6444.466906] Lustre: DEBUG MARKER: == sanity test 39b: mtime change on open, link, unlink, rename ========================================================== 18:55:44 (1787698544) [ 6452.983752] Lustre: DEBUG MARKER: == sanity test 39c: mtime change on rename =============== 18:55:53 (1787698553) [ 6463.240630] Lustre: DEBUG MARKER: == sanity test 39d: create, utime, stat ================== 18:56:03 (1787698563) [ 6470.670867] Lustre: DEBUG MARKER: == sanity test 39e: create, stat, utime, stat ============ 18:56:10 (1787698570) [ 6478.667902] Lustre: DEBUG MARKER: == sanity test 39f: create, stat, sleep, utime, stat ===== 18:56:18 (1787698578) [ 6488.593807] Lustre: DEBUG MARKER: == sanity test 39g: write, chmod, stat =================== 18:56:28 (1787698588) [ 6497.475675] Lustre: DEBUG MARKER: == sanity test 39h: write, utime within one second, stat ========================================================== 18:56:37 (1787698597) [ 6506.168971] Lustre: DEBUG MARKER: == sanity test 39i: write, rename, stat ================== 18:56:46 (1787698606) [ 6513.973543] Lustre: DEBUG MARKER: == sanity test 39j: write, rename, close, stat =========== 18:56:53 (1787698613) [ 6544.007744] Lustre: DEBUG MARKER: == sanity test 39k: write, utime, close, stat ============ 18:57:23 (1787698643) [ 6554.380821] Lustre: DEBUG MARKER: == sanity test 39l: directory atime update =============== 18:57:34 (1787698654) [ 6572.911323] Lustre: DEBUG MARKER: == sanity test 39m: test atime and mtime before 1970 ===== 18:57:52 (1787698672) [ 6581.072204] Lustre: DEBUG MARKER: == sanity test 39n: check that O_NOATIME is honored ====== 18:58:00 (1787698680) [ 6601.346783] Lustre: DEBUG MARKER: == sanity test 39o: directory cached attributes updated after create ========================================================== 18:58:21 (1787698701) [ 6609.666947] Lustre: DEBUG MARKER: == sanity test 39p: remote directory cached attributes updated after create ================================================================== 18:58:29 (1787698709) [ 6617.829651] Lustre: DEBUG MARKER: == sanity test 39r: lazy atime update on OST ============= 18:58:37 (1787698717) [ 6641.898267] Lustre: DEBUG MARKER: == sanity test 39q: close won't zero out atime =========== 18:59:01 (1787698741) [ 6648.925402] Lustre: DEBUG MARKER: == sanity test 39s: relatime is supported ================ 18:59:08 (1787698748) [ 6671.633567] Lustre: DEBUG MARKER: == sanity test 39u: stat race ============================ 18:59:31 (1787698771) [ 6692.794402] Lustre: DEBUG MARKER: == sanity test 40: failed open(O_TRUNC) doesn't truncate ======================================================================= 18:59:52 (1787698792) [ 6699.095761] Lustre: DEBUG MARKER: == sanity test 41: test small file write + fstat =============================================================================== 18:59:59 (1787698799) [ 6705.270723] Lustre: DEBUG MARKER: SKIP: sanity test_42a skipping ALWAYS excluded test 42a [ 6706.651637] Lustre: DEBUG MARKER: SKIP: sanity test_42b skipping ALWAYS excluded test 42b [ 6708.095561] Lustre: DEBUG MARKER: SKIP: sanity test_42c skipping ALWAYS excluded test 42c [ 6709.579521] Lustre: DEBUG MARKER: == sanity test 42d: test complete truncate of file with cached dirty data ========================================================== 19:00:09 (1787698809) [ 6718.451316] Lustre: DEBUG MARKER: == sanity test 42e: verify sub-RPC writes are not done synchronously ========================================================== 19:00:18 (1787698818) [ 6955.962241] Lustre: DEBUG MARKER: == sanity test 43A: execution of file opened for write should return -ETXTBSY ========================================================== 19:04:16 (1787699056) [ 6964.239425] Lustre: DEBUG MARKER: == sanity test 43a: open(RDWR) of file being executed should return -ETXTBSY ========================================================== 19:04:24 (1787699064) [ 6972.770774] Lustre: DEBUG MARKER: == sanity test 43b: truncate of file being executed should return -ETXTBSY ========================================================== 19:04:32 (1787699072) [ 6980.891702] Lustre: DEBUG MARKER: == sanity test 43c: md5sum of copy into lustre =========== 19:04:40 (1787699080) [ 6987.916498] Lustre: DEBUG MARKER: == sanity test 44A: zero length read from a sparse stripe ========================================================== 19:04:47 (1787699087) [ 6994.419785] Lustre: DEBUG MARKER: == sanity test 44a: test sparse pwrite ========================================================================================= 19:04:54 (1787699094) [ 7005.946191] Lustre: DEBUG MARKER: == sanity test 44b: write one byte at offset 0xfffffffe000 ========================================================== 19:05:05 (1787699105) [ 7012.954412] Lustre: DEBUG MARKER: == sanity test 44c: write 1 byte at max_object_bytes - 1 offset ========================================================== 19:05:12 (1787699112) [ 7018.993075] Lustre: DEBUG MARKER: == sanity test 44d: if write at position fails (EFBIG), so should do append ========================================================== 19:05:19 (1787699119) [ 7025.484142] Lustre: DEBUG MARKER: == sanity test 44e: write and read maximal stripes ======= 19:05:25 (1787699125) [ 7051.333943] Lustre: DEBUG MARKER: == sanity test 44f: Check fiemap for sparse files ======== 19:05:51 (1787699151) [ 7430.523516] LustreError: 101756:0:(tgt_grant.c:747:tgt_grant_check()) lustre-OST0001: cli b297afa7-61cb-4b6d-97e7-735874c77f8d claims 2121728 GRANT, real grant 0 [ 7574.948978] LustreError: 101758:0:(tgt_grant.c:747:tgt_grant_check()) lustre-OST0000: cli b297afa7-61cb-4b6d-97e7-735874c77f8d claims 2121728 GRANT, real grant 0 [ 7652.259873] Lustre: DEBUG MARKER: == sanity test 44g: test overflow in lov_stripe_size ===== 19:15:51 (1787699751) [ 7659.073956] Lustre: DEBUG MARKER: == sanity test 45: osc io page accounting ================ 19:15:59 (1787699759) [ 7666.418635] Lustre: DEBUG MARKER: == sanity test 46: dirtying a previously written page ========================================================================== 19:16:06 (1787699766) [ 7673.694255] Lustre: DEBUG MARKER: == sanity test 48a: Access renamed working dir (should return errors)=========================================================== 19:16:13 (1787699773) [ 7681.777661] Lustre: DEBUG MARKER: == sanity test 48b: Access removed working dir (should return errors)=========================================================== 19:16:21 (1787699781) [ 7689.073448] Lustre: DEBUG MARKER: == sanity test 48c: Access removed working subdir (should return errors) ========================================================== 19:16:29 (1787699789) [ 7696.550771] Lustre: DEBUG MARKER: == sanity test 48d: Access removed parent subdir (should return errors) ========================================================== 19:16:36 (1787699796) [ 7704.641774] Lustre: DEBUG MARKER: == sanity test 48e: Access to recreated parent subdir (should return errors) ========================================================== 19:16:44 (1787699804) [ 7712.543487] Lustre: DEBUG MARKER: == sanity test 48f: non-zero nlink dir unlink won't LBUG() ========================================================== 19:16:52 (1787699812) [ 7714.410879] Lustre: DEBUG MARKER: SKIP: sanity test_48f needs different host for mdt1 mdt2 [ 7716.210890] Lustre: DEBUG MARKER: == sanity test 49a: Change max_pages_per_rpc won't break osc extent ========================================================== 19:16:56 (1787699816) [ 8371.167825] Lustre: DEBUG MARKER: == sanity test 49b: verify max_mb_per_rpc_read/write after setting max_pages_per_rpc ========================================================== 19:27:51 (1787700471) [ 8378.249599] Lustre: DEBUG MARKER: == sanity test 49c: verify max_mb_per_rpc_read/write ===== 19:27:57 (1787700477) [ 8385.846562] Lustre: DEBUG MARKER: == sanity test 50: special situations: /proc symlinks ========================================================================= 19:28:05 (1787700485) [ 8394.369386] Lustre: DEBUG MARKER: == sanity test 51a: special situations: split htree with empty entry ============================================================ 19:28:13 (1787700493) [ 8409.387357] Lustre: DEBUG MARKER: == sanity test 51b: exceed 64k subdirectory nlink limit on create, verify unlink ========================================================== 19:28:29 (1787700509) [10312.876576] Lustre: DEBUG MARKER: SKIP: sanity test_51c skipping SLOW test 51c [10314.753556] Lustre: DEBUG MARKER: == sanity test 51d: check LOV round-robin OST object distribution ========================================================== 20:00:14 (1787702414) [10316.616600] Lustre: DEBUG MARKER: SKIP: sanity test_51d needs >= 3 OSTs [10318.487719] Lustre: DEBUG MARKER: SKIP: sanity test_51e skipping SLOW test 51e [10320.398569] Lustre: DEBUG MARKER: == sanity test 51f: check many open files limit ========== 20:00:20 (1787702420) [10630.268965] Lustre: DEBUG MARKER: == sanity test 52a: append-only flag test (should return errors) ========================================================== 20:05:30 (1787702730) [10637.016383] Lustre: DEBUG MARKER: == sanity test 52b: immutable flag test (should return errors) ================================================================= 20:05:36 (1787702736) [10643.865175] Lustre: DEBUG MARKER: == sanity test 53: verify that MDS and OSTs agree on pre-creation ============================================================== 20:05:43 (1787702743) [10657.534295] Lustre: DEBUG MARKER: == sanity test 54a: unix domain socket test ============== 20:05:57 (1787702757) [10665.156552] Lustre: DEBUG MARKER: == sanity test 54b: char device works in lustre ================================================================================ 20:06:05 (1787702765) [10672.119459] Lustre: DEBUG MARKER: == sanity test 54c: block device works in lustre =============================================================================== 20:06:11 (1787702771) [10680.896954] Lustre: DEBUG MARKER: == sanity test 54d: fifo device works in lustre ================================================================================ 20:06:20 (1787702780) [10687.046634] Lustre: DEBUG MARKER: == sanity test 54e: console/tty device works in lustre ================================================================================ 20:06:27 (1787702787) [10693.869889] Lustre: DEBUG MARKER: == sanity test 55a: OBD device life cycle unit tests ===== 20:06:33 (1787702793) [10700.549139] Lustre: DEBUG MARKER: == sanity test 55b: Load and unload max OBD devices ====== 20:06:40 (1787702800) [11360.296512] Lustre: DEBUG MARKER: == sanity test 55c: obd_mod_rpcs_test ==================== 20:17:39 (1787703459) [11367.254147] Lustre: DEBUG MARKER: == sanity test 56a: check /home/green/git/lustre-release/lustre/utils/lfs getstripe ========================================================== 20:17:47 (1787703467) [11376.950384] Lustre: DEBUG MARKER: == sanity test 56b: check /home/green/git/lustre-release/lustre/utils/lfs getdirstripe ========================================================== 20:17:56 (1787703476) [11385.598872] Lustre: DEBUG MARKER: == sanity test 56bb: check /home/green/git/lustre-release/lustre/utils/lfs getdirstripe layout is YAML ========================================================== 20:18:05 (1787703485) [11394.200859] Lustre: DEBUG MARKER: == sanity test 56bc: check '/home/green/git/lustre-release/lustre/utils/lfs getdirstripe --yaml' params are valid ========================================================== 20:18:13 (1787703493) [11401.638261] Lustre: DEBUG MARKER: == sanity test 56bd: check '/home/green/git/lustre-release/lustre/utils/lfs getdirstripe' output contains header only when needed ========================================================== 20:18:21 (1787703501) [11407.680622] Lustre: DEBUG MARKER: == sanity test 56c: check 'lfs df' showing device status ========================================================== 20:18:27 (1787703507) [11438.236800] Lustre: DEBUG MARKER: == sanity test 56ca: 'lfs df -v' correctly reports 'R' flag when OST set as Readonly ========================================================== 20:18:58 (1787703538) [11469.576585] Lustre: DEBUG MARKER: == sanity test 56d: 'lfs df -v' prints only configured devices ========================================================== 20:19:29 (1787703569) [11476.455476] Lustre: DEBUG MARKER: == sanity test 56e: 'lfs df' Handle non LustreFS [11483.265548] Lustre: DEBUG MARKER: == sanity test 56g: check lfs find -name ================= 20:19:43 (1787703583) [11492.311900] Lustre: DEBUG MARKER: == sanity test 56h: check lfs find ! -name =============== 20:19:52 (1787703592) [11498.589308] Lustre: DEBUG MARKER: == sanity test 56i: check 'lfs find -ost UUID' skips directories ========================================================== 20:19:58 (1787703598) [11504.657524] Lustre: DEBUG MARKER: == sanity test 56ib: check 'lfs find -ost INDEX_RANGE' command ========================================================== 20:20:04 (1787703604) [11510.841810] Lustre: DEBUG MARKER: == sanity test 56j: check lfs find -type d =============== 20:20:11 (1787703611) [11517.804461] Lustre: DEBUG MARKER: == sanity test 56k: check lfs find -type f =============== 20:20:17 (1787703617) [11525.077700] Lustre: DEBUG MARKER: == sanity test 56l: check lfs find -type b =============== 20:20:25 (1787703625) [11533.925646] Lustre: DEBUG MARKER: == sanity test 56m: check lfs find -type c =============== 20:20:33 (1787703633) [11541.457245] Lustre: DEBUG MARKER: == sanity test 56n: check lfs find -type l =============== 20:20:41 (1787703641) [11548.634410] Lustre: DEBUG MARKER: == sanity test 56o: check lfs find -mtime for old files == 20:20:48 (1787703648) [11558.945635] Lustre: DEBUG MARKER: == sanity test 56ob: check lfs find -atime -mtime -ctime with units ========================================================== 20:20:58 (1787703658) [11570.249562] Lustre: DEBUG MARKER: SKIP: sanity test_56oc skipping excluded test 56oc [11571.715141] Lustre: DEBUG MARKER: == sanity test 56od: check lfs find -btime with units ==== 20:21:11 (1787703671) [11587.155229] Lustre: DEBUG MARKER: == sanity test 56oe: check lfs find with time range ====== 20:21:27 (1787703687) [11721.279132] Lustre: DEBUG MARKER: == sanity test 56p: check lfs find -uid and ! -uid ======= 20:23:41 (1787703821) [11730.634987] Lustre: DEBUG MARKER: == sanity test 56q: check lfs find -gid and ! -gid ======= 20:23:50 (1787703830) [11739.832971] Lustre: DEBUG MARKER: == sanity test 56r: check lfs find -size works =========== 20:23:59 (1787703839) [11751.919541] Lustre: DEBUG MARKER: == sanity test 56ra: check lfs find -size -lazy works for data on OSTs ========================================================== 20:24:11 (1787703851) [11766.548639] Lustre: DEBUG MARKER: == sanity test 56rb: check lfs find --size --ost/--mdt works ========================================================== 20:24:26 (1787703866) [11774.539856] Lustre: DEBUG MARKER: == sanity test 56rc: check lfs find --mdt-count/--mdt-hash works ========================================================== 20:24:34 (1787703874) [11785.455812] Lustre: DEBUG MARKER: == sanity test 56rd: check lfs find --printf special files ========================================================== 20:24:45 (1787703885) [11792.971413] Lustre: DEBUG MARKER: == sanity test 56re: check lfs find -printf width format specifiers are consistant with regular find ========================================================== 20:24:52 (1787703892) [11804.330671] Lustre: DEBUG MARKER: == sanity test 56rf: check lfs find -printf width format specifiers for lustre specific formats ========================================================== 20:25:04 (1787703904) [11823.920268] Lustre: DEBUG MARKER: == sanity test 56s: check lfs find -stripe-count works === 20:25:23 (1787703923) [11833.656722] Lustre: DEBUG MARKER: == sanity test 56t: check lfs find -stripe-size works ==== 20:25:33 (1787703933) [11845.746939] Lustre: DEBUG MARKER: == sanity test 56u: check lfs find -stripe-index works === 20:25:45 (1787703945) [11855.684386] Lustre: DEBUG MARKER: == sanity test 56v: check 'lfs find -m match with lfs getstripe -m' ========================================================== 20:25:55 (1787703955) [11865.924047] Lustre: DEBUG MARKER: == sanity test 56wa: check 'lfs migrate -c stripe_count' works ========================================================== 20:26:05 (1787703965) [12101.425253] Lustre: DEBUG MARKER: == sanity test 56wb: check 'lfs migrate' pool support ==== 20:30:01 (1787704201) [12127.002257] Lustre: DEBUG MARKER: == sanity test 56wc: check unrecognized options for lfs migrate are passed through ========================================================== 20:30:27 (1787704227) [12159.058660] Lustre: DEBUG MARKER: == sanity test 56wd: check lfs_migrate --rsync and --no-rsync work ========================================================== 20:30:59 (1787704259) [12166.056534] Lustre: DEBUG MARKER: == sanity test 56we: check lfs migrate --non-direct|-D support ========================================================== 20:31:06 (1787704266) [12172.752610] Lustre: DEBUG MARKER: == sanity test 56x: lfs migration support ================ 20:31:12 (1787704272) [12180.494866] Lustre: DEBUG MARKER: == sanity test 56xB: lfs migrate with -0, --null, --files-from arguments ========================================================== 20:31:20 (1787704280) [12191.113745] Lustre: DEBUG MARKER: == sanity test 56xC: lfs migration can accept FID list file ========================================================== 20:31:31 (1787704291) [12200.768364] Lustre: DEBUG MARKER: == sanity test 56xD: lfs_migration can accept FIDs ======= 20:31:40 (1787704300) [12211.061776] Lustre: DEBUG MARKER: == sanity test 56xa: lfs migration --block support ======= 20:31:50 (1787704310) [12218.485751] Lustre: DEBUG MARKER: == sanity test 56xab: lfs migration --block on changing file ========================================================== 20:31:58 (1787704318) [12226.775562] Lustre: DEBUG MARKER: == sanity test 56xb: lfs migration hard link support ===== 20:32:06 (1787704326) [12477.929463] Lustre: DEBUG MARKER: == sanity test 56xc: lfs migration autostripe ============ 20:36:18 (1787704578) [12485.106138] Lustre: DEBUG MARKER: == sanity test 56xd: check lfs migrate --yaml and --copy support ========================================================== 20:36:25 (1787704585) [12499.444520] Lustre: DEBUG MARKER: == sanity test 56xe: migrate a composite layout file ===== 20:36:39 (1787704599) [12503.828959] Lustre: 6502:0:(mdt_recovery.c:102:mdt_req_from_lrd()) @@@ restoring transno req@ffff8bc2d7cae300 x1874531247677312/t25769980542(0) o101->b297afa7-61cb-4b6d-97e7-735874c77f8d@192.168.203.23@tcp:516/0 lens 376/816 e 0 to 0 dl 1787704616 ref 1 fl Interpret:H/602/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 [12504.543062] Lustre: 6500:0:(mdt_recovery.c:102:mdt_req_from_lrd()) @@@ restoring transno req@ffff8bc2f75d0a80 x1874531247679360/t25769980544(0) o101->b297afa7-61cb-4b6d-97e7-735874c77f8d@192.168.203.23@tcp:517/0 lens 376/840 e 0 to 0 dl 1787704617 ref 1 fl Interpret:H/602/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 [12504.570438] Lustre: 6500:0:(mdt_recovery.c:102:mdt_req_from_lrd()) Skipped 1 previous similar message [12516.709493] Lustre: DEBUG MARKER: == sanity test 56xf: FID is not lost during migration of a composite layout file ========================================================== 20:36:56 (1787704616) [12527.426874] Lustre: DEBUG MARKER: == sanity test 56xg: lfs migrate pool support ============ 20:37:07 (1787704627) [12592.517140] Lustre: DEBUG MARKER: == sanity test 56xh: lfs migrate bandwidth limitation support ========================================================== 20:38:12 (1787704692) [12630.775726] Lustre: DEBUG MARKER: == sanity test 56xi: lfs migrate stats support =========== 20:38:50 (1787704730) [12643.005989] Lustre: DEBUG MARKER: == sanity test 56xj: lfs migrate -b should not cause starvation of threads on OSS ========================================================== 20:39:02 (1787704742) [12680.472176] Lustre: DEBUG MARKER: == sanity test 56xk: lfs mirror resync bandwidth limitation support ========================================================== 20:39:40 (1787704780) [12693.162185] Lustre: DEBUG MARKER: == sanity test 56xl: lfs mirror resync stats support ===== 20:39:53 (1787704793) [12701.388591] Lustre: DEBUG MARKER: == sanity test 56y: lfs find -L raid0|released =========== 20:40:01 (1787704801) [12707.205924] Lustre: DEBUG MARKER: == sanity test 56z: lfs find should continue after an error ========================================================== 20:40:07 (1787704807) [12717.161542] Lustre: DEBUG MARKER: == sanity test 56Aa: lfs find --size under striped dir === 20:40:17 (1787704817) [12761.001410] Lustre: DEBUG MARKER: == sanity test 56ab: lfs find --blocks =================== 20:41:01 (1787704861) [12771.909345] Lustre: DEBUG MARKER: == sanity test 56aca: check lfs find -perm with octal representation ========================================================== 20:41:11 (1787704871) [12811.084721] Lustre: DEBUG MARKER: == sanity test 56acb: check lfs find -perm with symbolic representation ========================================================== 20:41:51 (1787704911) [12825.263949] Lustre: DEBUG MARKER: == sanity test 56acc: check parsing error for lfs find -perm ========================================================== 20:42:05 (1787704925) [12831.895753] Lustre: DEBUG MARKER: == sanity test 56Ba: test lfs find --component-end, -start, -count, and -flags ========================================================== 20:42:11 (1787704931) [12845.803400] Lustre: DEBUG MARKER: == sanity test 56Ca: check lfs find --mirror-count|-N and --mirror-state ========================================================== 20:42:25 (1787704945) [12857.997578] Lustre: DEBUG MARKER: == sanity test 56Da: test lfs find with long paths ======= 20:42:37 (1787704957) [12876.708691] Lustre: DEBUG MARKER: == sanity test 56Db: test 'lfs df -m' only shows MDT devices ========================================================== 20:42:56 (1787704976) [12883.054488] Lustre: DEBUG MARKER: == sanity test 56Dc: test 'lfs df -o' only shows OST devices ========================================================== 20:43:03 (1787704983) [12889.359950] Lustre: DEBUG MARKER: == sanity test 56Dd: test lfs find with mindepth argument ========================================================== 20:43:09 (1787704989) [12895.180518] Lustre: DEBUG MARKER: == sanity test 56Ea: test lfs find -printf option ======== 20:43:15 (1787704995) [12921.591467] Lustre: DEBUG MARKER: == sanity test 56Eaa: test lfs find -printf added functions ========================================================== 20:43:41 (1787705021) [13126.126641] Lustre: DEBUG MARKER: == sanity test 56Eab: test lfs find -ls function ========= 20:47:06 (1787705226) [13145.483166] Lustre: DEBUG MARKER: == sanity test 56Eb: check lfs getstripe on symlink ====== 20:47:25 (1787705245) [13153.160983] Lustre: DEBUG MARKER: == sanity test 56Ebb: check /home/green/git/lustre-release/lustre/utils/lfs getdirstripe for FIFO file ========================================================== 20:47:33 (1787705253) [13159.784497] Lustre: DEBUG MARKER: == sanity test 56Ec: check lfs getstripe,setstripe --hex --yaml ========================================================== 20:47:39 (1787705259) [13165.956441] Lustre: DEBUG MARKER: == sanity test 56Ed: verify new YAML format is valid and back-compatible ========================================================== 20:47:46 (1787705266) [13187.929339] Lustre: DEBUG MARKER: == sanity test 56Eda: check lfs find --links ============= 20:48:08 (1787705288) [13194.953764] Lustre: DEBUG MARKER: == sanity test 56Edb: check lfs find --links for directory striped on multiple MDTs ========================================================== 20:48:15 (1787705295) [13201.790499] Lustre: DEBUG MARKER: == sanity test 56Ef: lfs find with multiple paths ======== 20:48:21 (1787705301) [13208.753101] Lustre: DEBUG MARKER: == sanity test 56Eg: lfs find -xattr ===================== 20:48:28 (1787705308) [13217.577900] Lustre: DEBUG MARKER: == sanity test 56Eh: check lfs find --skip =============== 20:48:37 (1787705317) [13268.557436] Lustre: DEBUG MARKER: == sanity test 56Ei: test lfs find --printf prints correct projid for special files ========================================================== 20:49:28 (1787705368) [13276.284339] Lustre: DEBUG MARKER: == sanity test 56Ej: lfs migration --non-block copy ====== 20:49:36 (1787705376) [13283.576326] Lustre: DEBUG MARKER: == sanity test 56Ek: Test lfs find error handling with LLAPI_FAIL_LOC ========================================================== 20:49:43 (1787705383) [13291.844435] Lustre: DEBUG MARKER: == sanity test 57a: verify MDS filesystem created with large inodes ============================================================ 20:49:51 (1787705391) [13301.356351] Lustre: DEBUG MARKER: == sanity test 57b: default LOV EAs are stored inside large inodes ============================================================= 20:50:01 (1787705401) [13318.948239] Lustre: DEBUG MARKER: == sanity test 58: verify cross-platform wire constants ======================================================================== 20:50:19 (1787705419) [13324.699324] Lustre: DEBUG MARKER: == sanity test 59: verify cancellation of llog records async =================================================================== 20:50:24 (1787705424) [13373.304282] Lustre: DEBUG MARKER: == sanity test complete, duration 13129 sec ============== 20:51:12 (1787705472) [13375.142507] Lustre: DEBUG MARKER: === sanity: start cleanup 20:51:14 (1787705474) === [13553.450934] Lustre: DEBUG MARKER: === sanity: finish cleanup 20:54:13 (1787705653) === [13555.708892] Lustre: Failing over lustre-MDT0000 [13556.192400] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13556.199372] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13556.211514] Lustre: Skipped 3 previous similar messages [13556.224503] LustreError: 69447:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13556.240159] LustreError: 69447:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [13556.883703] Lustre: server umount lustre-MDT0000 complete [13562.337183] LustreError: 107668:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13562.357032] LustreError: 107668:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [13572.578869] LustreError: 104097:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13572.609647] LustreError: 104097:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [13573.599243] Lustre: 3632:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787705659/real 1787705659] req@ffff8bc2ee9d5500 x1874531118292096/t0(0) o400->MGC192.168.203.123@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787705675 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [13573.634850] LustreError: MGC192.168.203.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13582.698808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13583.840408] LustreError: 137186:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [13583.846871] LustreError: 3628:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8bc1ccc87b80 x1874531118301440/t0(0) o250->MGC192.168.203.123@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [13584.141945] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13584.228211] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [13585.274235] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [13585.297318] Lustre: Skipped 1 previous similar message [13589.006339] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [13589.491578] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [13589.499089] Lustre: Skipped 3 previous similar messages [13589.689718] Lustre: lustre-MDT0000: Recovery over after 0:04, of 2 clients 2 recovered and 0 were evicted. [13589.695706] Lustre: Skipped 1 previous similar message [13589.763661] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000402:8514 to 0x280000402:8577) [13589.773950] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000402:8558 to 0x2c0000402:8641) [13597.261865] Lustre: DEBUG MARKER: oleg323-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13598.731392] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [13604.832251] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13604.837485] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13604.843590] Lustre: Skipped 6 previous similar messages [13604.864137] Lustre: Skipped 3 previous similar messages [13609.952017] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13609.954813] Lustre: Skipped 3 previous similar messages [13610.278619] Lustre: server umount lustre-MDT0000 complete [13615.074773] LustreError: 90798:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13615.091370] LustreError: 90798:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 16 previous similar messages [13617.586766] LustreError: 27304:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787705719 with bad export cookie 811737868692804786 [13617.591867] LustreError: MGC192.168.203.123@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13617.597416] LustreError: 27304:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13617.949105] Lustre: server umount lustre-MDT0001 complete [13635.575957] Lustre: server umount lustre-OST0000 complete [13636.258232] Lustre: 3630:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787705722/real 1787705722] req@ffff8bc2f6ce7100 x1874531118354560/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787705738 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [13636.308601] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13638.634271] Lustre: 3632:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787705724/real 1787705724] req@ffff8bc2f3074000 x1874531118354816/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787705740 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [13641.695343] Lustre: 3629:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787705727/real 1787705727] req@ffff8bc2f3077100 x1874531118355072/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787705743 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [13646.426785] Lustre: server umount lustre-OST0001 complete [13662.972777] Lustre: DEBUG MARKER: oleg323-server.virtnet: executing unload_modules_local [13665.852404] Key type lgssc unregistered [13666.091212] LNet: 140119:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13666.099515] LNetError: 140119:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13666.120562] LNet: Removed LNI 192.168.203.123@tcp [13667.180039] Key type .llcrypt unregistered [13667.183834] Key type ._llcrypt unregistered