[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-22) (GCC)) #7 SMP Sat Jan 18 21:01:29 EST 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0.0 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 485713358 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5410-0x000f541f] [ 0.000000] RAMDISK: [mem 0xbcbf3000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5220 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1D6F 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1C0B 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001BCB (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1C7F 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1D0F 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1D47 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1c0b-0xbffe1c7e] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1c0a] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1c7f-0xbffe1d0e] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1d0f-0xbffe1d46] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1d47-0xbffe1d6e] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 513 pages/cpu s2064384 r8192 d28672 u4194304 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2894836K/4306400K available (20483K kernel code, 12066K rwdata, 7356K rodata, 4680K init, 23504K bss, 542472K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41388 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x616/0x99a with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar [ 0.001000] ... MAX_LOCKDEP_SUBCLASSES: 8 [ 0.001000] ... MAX_LOCK_DEPTH: 48 [ 0.001000] ... MAX_LOCKDEP_KEYS: 8192 [ 0.001000] ... CLASSHASH_SIZE: 4096 [ 0.001000] ... MAX_LOCKDEP_ENTRIES: 32768 [ 0.001000] ... MAX_LOCKDEP_CHAINS: 65536 [ 0.001000] ... CHAINHASH_SIZE: 32768 [ 0.001000] memory used by lock dependency info: 4149 kB [ 0.001000] per task-struct memory footprint: 2688 bytes [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001016] APIC: Switch to symmetric I/O mode setup [ 0.002421] x2apic enabled [ 0.003011] Switched APIC routing to physical x2apic. [ 0.004021] kvm-guest: setup PV IPIs [ 0.007873] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008038] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009021] pid_max: default: 32768 minimum: 301 [ 0.011155] LSM: Security Framework initializing [ 0.013073] Yama: becoming mindful. [ 0.014128] SELinux: Initializing. [ 0.015284] *** VALIDATE selinux *** [ 0.023337] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028329] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.031087] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032147] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.033287] *** VALIDATE tmpfs *** [ 0.035434] *** VALIDATE proc *** [ 0.037625] *** VALIDATE cgroup *** [ 0.038023] *** VALIDATE cgroup2 *** [ 0.039381] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.040196] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.041000] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.042011] Spectre V2 : User space: Vulnerable [ 0.043011] Speculative Store Bypass: Vulnerable [ 0.045962] debug: unmapping init [mem 0xffffffffbaf03000-0xffffffffbaf0afff] [ 0.048845] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.051098] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.052047] ... version: 2 [ 0.053018] ... bit width: 48 [ 0.054016] ... generic registers: 4 [ 0.055017] ... value mask: 0000ffffffffffff [ 0.056020] ... max period: 00007fffffffffff [ 0.057019] ... fixed-purpose events: 3 [ 0.058018] ... event mask: 000000070000000f [ 0.059556] rcu: Hierarchical SRCU implementation. [ 0.064396] smp: Bringing up secondary CPUs ... [ 0.066815] x86: Booting SMP configuration: [ 0.067027] .... node #0, CPUs: #1 [ 0.071910] #2 [ 0.075842] #3 [ 0.078047] smp: Brought up 1 node, 4 CPUs [ 0.079048] smpboot: Max logical packages: 1 [ 0.080033] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.189155] node 0 deferred pages initialised in 103ms [ 0.190240] pgdatinit0 (35) used greatest stack depth: 14528 bytes left [ 0.196017] devtmpfs: initialized [ 0.198123] x86/mm: Memory block size: 128MB [ 0.211714] gcov: version magic: 0x41383552 [ 0.214824] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.218281] futex hash table entries: 1024 (order: 5, 131072 bytes, vmalloc) [ 0.221803] pinctrl core: initialized pinctrl subsystem [ 0.223873] [ 0.225021] ************************************************************* [ 0.227018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.229019] ** ** [ 0.230000] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.230000] ** ** [ 0.232019] ** This means that this kernel is built to expose internal ** [ 0.234017] ** IOMMU data structures, which may compromise security on ** [ 0.236071] ** your system. ** [ 0.239019] ** ** [ 0.241017] ** If you see this message and you are not debugging the ** [ 0.243019] ** kernel, report this immediately to your vendor! ** [ 0.246017] ** ** [ 0.248014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.250019] ************************************************************* [ 0.255131] NET: Registered protocol family 16 [ 0.258453] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.261122] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.264140] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.270311] cpuidle: using governor menu [ 0.273000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.275635] PCI: Using configuration type 1 for base access [ 0.278161] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.328430] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.330019] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.346689] cryptd: max_cpu_qlen set to 1000 [ 0.353164] ACPI: Added _OSI(Module Device) [ 0.354019] ACPI: Added _OSI(Processor Device) [ 0.355017] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.356023] ACPI: Added _OSI(Processor Aggregator Device) [ 0.396672] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.409487] ACPI: Interpreter enabled [ 0.411254] ACPI: PM: (supports S0 S3 S4 S5) [ 0.412018] ACPI: Using IOAPIC for interrupt routing [ 0.414284] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.420361] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.498082] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.499073] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.500025] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.501381] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.510281] acpiphp: Slot [2] registered [ 0.511387] acpiphp: Slot [5] registered [ 0.512445] acpiphp: Slot [6] registered [ 0.513383] acpiphp: Slot [3] registered [ 0.514336] acpiphp: Slot [4] registered [ 0.515284] acpiphp: Slot [7] registered [ 0.516333] acpiphp: Slot [8] registered [ 0.517323] acpiphp: Slot [9] registered [ 0.518313] acpiphp: Slot [10] registered [ 0.519328] acpiphp: Slot [11] registered [ 0.520311] acpiphp: Slot [12] registered [ 0.521358] acpiphp: Slot [13] registered [ 0.522349] acpiphp: Slot [14] registered [ 0.523377] acpiphp: Slot [15] registered [ 0.524289] acpiphp: Slot [16] registered [ 0.525307] acpiphp: Slot [17] registered [ 0.526325] acpiphp: Slot [18] registered [ 0.527302] acpiphp: Slot [19] registered [ 0.528332] acpiphp: Slot [20] registered [ 0.529337] acpiphp: Slot [21] registered [ 0.530329] acpiphp: Slot [22] registered [ 0.531286] acpiphp: Slot [23] registered [ 0.532373] acpiphp: Slot [24] registered [ 0.533319] acpiphp: Slot [25] registered [ 0.534317] acpiphp: Slot [26] registered [ 0.535346] acpiphp: Slot [27] registered [ 0.536338] acpiphp: Slot [28] registered [ 0.537307] acpiphp: Slot [29] registered [ 0.538329] acpiphp: Slot [30] registered [ 0.539355] acpiphp: Slot [31] registered [ 0.540164] PCI host bridge to bus 0000:00 [ 0.541029] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.542045] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.543035] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.544043] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.545073] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x38007fffffff window] [ 0.546059] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.547436] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.550247] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.553801] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.557903] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.559892] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.560022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.561021] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.562017] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.564828] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.565856] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.566052] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.568981] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.570020] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.575997] pci 0000:00:02.0: reg 0x20: [mem 0x380000000000-0x380000003fff 64bit pref] [ 0.578019] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.596565] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.599021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.602020] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.609019] pci 0000:00:05.0: reg 0x20: [mem 0x380000004000-0x380000007fff 64bit pref] [ 0.629979] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.632019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.635024] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.642019] pci 0000:00:06.0: reg 0x20: [mem 0x380000008000-0x38000000bfff 64bit pref] [ 0.665493] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.670011] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.674000] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.678071] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.680922] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.690922] iommu: Default domain type: Passthrough [ 0.693270] SCSI subsystem initialized [ 0.694558] ACPI: bus type USB registered [ 0.695444] usbcore: registered new interface driver usbfs [ 0.696341] usbcore: registered new interface driver hub [ 0.697215] usbcore: registered new device driver usb [ 0.698732] pps_core: LinuxPPS API ver. 1 registered [ 0.699014] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.700153] PTP clock support registered [ 0.702394] EDAC MC: Ver: 3.0.0 [ 0.705208] PCI: Using ACPI for IRQ routing [ 0.708428] NetLabel: Initializing [ 0.709021] NetLabel: domain hash size = 128 [ 0.710016] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.711344] NetLabel: unlabeled traffic allowed by default [ 0.712578] vgaarb: loaded [ 0.714488] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.715042] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.729087] clocksource: Switched to clocksource kvm-clock [ 1.176819] VFS: Disk quotas dquot_6.6.0 [ 1.178538] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.181226] *** VALIDATE ramfs *** [ 1.182542] *** VALIDATE hugetlbfs *** [ 1.184472] pnp: PnP ACPI init [ 1.193234] pnp: PnP ACPI: found 6 devices [ 1.237840] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.241795] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.244078] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.246032] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.248433] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.250952] pci_bus 0000:00: resource 8 [mem 0x380000000000-0x38007fffffff window] [ 1.254798] NET: Registered protocol family 2 [ 1.258122] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.264129] tcp_listen_portaddr_hash hash table entries: 4096 (order: 6, 360448 bytes, vmalloc) [ 1.268113] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.274571] TCP bind hash table entries: 65536 (order: 10, 5242880 bytes, vmalloc) [ 1.282829] TCP: Hash tables configured (established 65536 bind 65536) [ 1.286218] MPTCP token hash table entries: 8192 (order: 7, 786432 bytes, vmalloc) [ 1.290470] UDP hash table entries: 4096 (order: 7, 786432 bytes, vmalloc) [ 1.294603] UDP-Lite hash table entries: 4096 (order: 7, 786432 bytes, vmalloc) [ 1.298841] NET: Registered protocol family 1 [ 1.302966] RPC: Registered named UNIX socket transport module. [ 1.305412] RPC: Registered udp transport module. [ 1.307159] RPC: Registered tcp transport module. [ 1.308942] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.311462] NET: Registered protocol family 44 [ 1.313132] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.315216] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.317131] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.319020] PCI: CLS 0 bytes, default 64 [ 1.320920] Unpacking initramfs... [ 3.396160] debug: unmapping init [mem 0xffff926b7cbf3000-0xffff926b7ffcffff] [ 3.400616] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.402915] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.405868] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.417320] cryptomgr_test (65) used greatest stack depth: 14248 bytes left [ 4.255560] Initialise system trusted keyrings [ 4.257380] Key type blacklist registered [ 4.259536] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 4.323561] zbud: loaded [ 4.339789] *** VALIDATE nfs *** [ 4.340824] *** VALIDATE nfs4 *** [ 4.342560] pstore: using deflate compression [ 4.348642] Platform Keyring initialized [ 4.353955] cryptomgr_test (73) used greatest stack depth: 14024 bytes left [ 4.379353] cryptomgr_test (86) used greatest stack depth: 13800 bytes left [ 4.411747] cryptomgr_test (94) used greatest stack depth: 13640 bytes left [ 4.544949] NET: Registered protocol family 38 [ 4.546837] Key type asymmetric registered [ 4.548605] Asymmetric key parser 'x509' registered [ 4.550513] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 4.554352] io scheduler mq-deadline registered [ 4.556520] io scheduler kyber registered [ 4.558809] io scheduler bfq registered [ 4.561107] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.566817] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.571097] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.574873] ACPI: Power Button [PWRF] [ 5.972064] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 7.360267] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 8.755093] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 8.789920] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 8.824050] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 8.836685] Non-volatile memory driver v1.3 [ 8.838830] Linux agpgart interface v0.103 [ 8.955988] virtio_blk virtio1: [vda] 131896 512-byte logical blocks (67.5 MB/64.4 MiB) [ 8.959297] vda: detected capacity change from 0 to 67530752 [ 8.993890] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 8.997238] vdb: detected capacity change from 0 to 1073741824 [ 9.015876] libphy: Fixed MDIO Bus: probed [ 9.030189] usbcore: registered new interface driver usbserial_generic [ 9.033856] usbserial: USB Serial support registered for generic [ 9.036928] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 9.042053] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 9.043892] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 9.048180] mousedev: PS/2 mouse device common for all mice [ 9.053456] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 9.054345] rtc_cmos 00:05: RTC can wake from S4 [ 9.062065] rtc_cmos 00:05: registered as rtc0 [ 9.062582] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 9.064277] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 9.069775] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 9.074324] intel_pstate: CPU model not supported [ 9.083352] hid: raw HID events driver (C) Jiri Kosina [ 9.085832] usbcore: registered new interface driver usbhid [ 9.087581] usbhid: USB HID core driver [ 9.089331] drop_monitor: Initializing network drop monitor service [ 9.092269] Initializing XFRM netlink socket [ 9.097167] NET: Registered protocol family 10 [ 9.105281] Segment Routing with IPv6 [ 9.107251] NET: Registered protocol family 17 [ 9.109585] mpls_gso: MPLS GSO support [ 9.118279] RAS: Correctable Errors collector initialized. [ 9.120040] AVX version of gcm_enc/dec engaged. [ 9.122742] AES CTR mode by8 optimization enabled [ 9.314658] sched_clock: Marking stable (9314590025, 0)->(10443288175, -1128698150) [ 9.319139] registered taskstats version 1 [ 9.321649] Loading compiled-in X.509 certificates [ 9.324790] zswap: loaded using pool lzo/zbud [ 9.387652] Key type big_key registered [ 9.416797] Key type encrypted registered [ 9.418570] ima: No TPM chip found, activating TPM-bypass! [ 9.420755] ima: Allocated hash algorithm: sha1 [ 9.422315] ima: No architecture policies found [ 9.424238] evm: Initialising EVM extended attributes: [ 9.426174] evm: security.selinux [ 9.427289] evm: security.ima [ 9.428289] evm: security.capability [ 9.429510] evm: HMAC attrs: 0x1 [ 9.437332] rtc_cmos 00:05: setting system clock to 2025-04-01 08:24:25 UTC (1743495865) [ 9.472489] debug: unmapping init [mem 0xffffffffbc403000-0xffffffffbc5fffff] [ 9.475763] debug: unmapping init [mem 0xffffffffbaa71000-0xffffffffbaf02fff] [ 9.484174] Write protecting the kernel read-only data: 30720k [ 9.487676] debug: unmapping init [mem 0xffffffffb9003000-0xffffffffb91fffff] [ 9.490719] debug: unmapping init [mem 0xffffffffb992f000-0xffffffffb99fffff] [ 9.582957] systemd[1]: systemd 239 (239-82.el8_10.3) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 9.590957] systemd[1]: Detected virtualization kvm. [ 9.592503] systemd[1]: Detected architecture x86-64. [ 9.594329] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 9.636131] systemd[1]: No hostname configured. [ 9.638088] systemd[1]: Set hostname to . [ 9.640586] random: systemd: uninitialized urandom read (16 bytes read) [ 9.643140] systemd[1]: Initializing machine ID from random generator. [ 9.794496] random: ln: uninitialized urandom read (6 bytes read) [ 9.976115] dracut-rootfs-g (197) used greatest stack depth: 13608 bytes left [ 10.025827] random: systemd: uninitialized urandom read (16 bytes read) [ 10.030755] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 10.060750] systemd[1]: Starting Setup Virtual Console... Starting Setup Virtual Console... [ 10.072670] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Slices. [ OK ] Reached target Timers. [ OK ] Reached target Swap. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Initrd Root Device. [ 10.404854] memstrack-start (236) used greatest stack depth: 13528 bytes left [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] [ 10.807373] random: fast init done Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 11.864404] device-mapper: uevent: version 1.0.3 [ 11.867958] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... [ 12.944560] udevadm (418) used greatest stack depth: 13344 bytes left Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 14.371409] virtio_net virtio0 ens2: renamed from eth0 [ 14.637425] scsi host0: ata_piix [ 14.850850] scsi host1: ata_piix [ 14.905205] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 14.907834] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 15.415425] systemd-udevd (449) used greatest stack depth: 13048 bytes left [ 15.514637] systemd-udevd (450) used greatest stack depth: 12536 bytes left [ 16.123593] ip (531) used greatest stack depth: 11496 bytes left [ 18.941517] random: crng init done [ 18.943086] random: 7 urandom warning(s) missed due to ratelimiting [ 20.247436] dracut-initqueue[593]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 21.836059] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 24.811443] printk: systemd: 25 output lines suppressed due to ratelimiting [ 25.355644] SELinux: Disabled at runtime. [ 25.451209] systemd[1]: systemd 239 (239-82.el8_10.3) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 25.459863] systemd[1]: Detected virtualization kvm. [ 25.461825] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 26.788810] systemd[1]: initrd-switch-root.service: Succeeded. [ 26.796446] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 26.801277] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 26.805610] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 26.809203] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 26.832614] systemd[1]: Starting Journal Service... Starting Journal Service... [ 26.857505] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Control Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Apply Kernel Variables... [ 27.003294] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice User and Session Slice. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Huge Pages File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice system-getty.slice. Mounting POSIX Message Queue File System... Starting udev Coldplug all Devices... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target rpc_pipefs.target. Starting Remount Root and Kernel File Systems... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Reached target Slices. Mounting Kernel Debug File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Mounted /mnt. [ 29.379424] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ OK ] Started udev Coldplug all Devices. [ 31.922559] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 32.163894] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 33.116528] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 33.375364] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (9s / no limit) [** ] A start job is running for Configur…only root support (11s / no limit) [*** ] A start job is running for Configur…only root support (11s / no limit) [ *** ] A start job is running for Configur…only root support (12s / no limit) [ *** ] A start job is running for Configur…only root support (12s / no limit) [ ***] A start job is running for Configur…only root support (13s / no limit) [ **] A start job is running for Configur…only root support (13s / no limit) [ *] A start job is running for Configur…only root support (14s / no limit) [ **] A start job is running for Configur…only root support (14s / no limit) [ ***] A start job is running for Configur…only root support (15s / no limit) [ *** ] A start job is running for Configur…only root support (15s / no limit) [ *** ] A start job is running for Configur…only root support (16s / no limit)[ 43.813220] Key type dns_resolver registered [*** ] A start job is running for Configur…only root support (17s / no limit)[ 44.073019] hrtimer: interrupt took 7005657 ns [** ] A start job is running for Configur…only root support (17s / no limit)[ 44.695941] NFS: Registering the id_resolver key type [ 44.703668] Key type id_resolver registered [ 44.715967] Key type id_legacy registered [* ] A start job is running for Configur…only root support (18s / no limit) [** ] A start job is running for Configur…only root support (18s / no limit)[ 45.541897] mount.nfs (968) used greatest stack depth: 10760 bytes left [*** ] A start job is running for Configur…only root support (19s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Load/Save Random Seed. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Network Manager... Starting Login Service... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. Starting Hostname Service... [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg607-client login: [ 82.086817] mount.nfs (1523) used greatest stack depth: 10456 bytes left [ 177.959543] libcfs: loading out-of-tree module taints kernel. [ 178.198628] Key type ._llcrypt registered [ 178.199766] Key type .llcrypt registered [ 179.027363] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 179.084821] alg: No test for adler32 (adler32-zlib) [ 181.061656] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 182.707248] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 182.710357] LNet: Accept secure, port 988 [ 184.906781] Key type lgssc registered [ 188.054909] Lustre: Echo OBD driver; http://www.lustre.org/ [ 507.261490] Lustre: Mounted lustre-client [ 516.919963] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 532.960892] Lustre: lustre-OST0000-osc-ffff926bd4a8c000: disconnect after 22s idle [ 546.381681] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing check_logdir /tmp/testlogs/ [ 556.087302] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing yml_node [ 568.313620] Lustre: DEBUG MARKER: Client: 2.16.52.73 [ 577.036654] Lustre: DEBUG MARKER: MDS: 2.16.52.73 [ 582.978483] Lustre: DEBUG MARKER: OSS: 2.16.52.73 [ 586.473942] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Apr 1 04:33:59 EDT 2025 [ 617.019176] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 620.533615] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 625.049869] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 635.452438] Lustre: Unmounted lustre-client [ 822.855988] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 830.851272] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 867.023575] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 925.102951] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 04:39:37 (1743496777) [ 980.989233] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 988.385617] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1022.003852] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1023.587779] Lustre: Mounted lustre-client [ 1034.739075] Lustre: Unmounted lustre-client [ 1221.472362] Lustre: Mounted lustre-client [ 1230.533836] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1247.201871] Lustre: lustre-OST0000-osc-ffff926bd13f4000: disconnect after 22s idle [ 1247.216360] Lustre: Skipped 1 previous similar message [ 1258.096354] Lustre: Unmounted lustre-client [ 1433.270021] Lustre: Mounted lustre-client [ 1442.812444] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1458.664414] Lustre: lustre-OST0000-osc-ffff926bd4b9c000: disconnect after 21s idle [ 1458.677511] Lustre: Skipped 1 previous similar message [ 1460.059154] Lustre: Unmounted lustre-client [ 1517.409255] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 04:49:30 (1743497370) [ 1666.108436] Lustre: Mounted lustre-client [ 1675.437525] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1690.221960] Lustre: Unmounted lustre-client [ 1744.456931] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 04:53:17 (1743497597) [ 1866.692396] Lustre: Mounted lustre-client [ 1874.836645] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1892.321944] Lustre: lustre-OST0000-osc-ffff926bc9bf4000: disconnect after 23s idle [ 1892.330470] Lustre: Skipped 1 previous similar message [ 1892.343849] Lustre: lustre-MDT0000-mdc-ffff926bc9bf4000: Connection to lustre-MDT0000 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 1902.574213] Lustre: lustre-MDT0001-mdc-ffff926bc9bf4000: Connection to lustre-MDT0001 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 1913.696151] Lustre: 2429:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743497753/real 1743497753] req@ffff926bc99350c0 x1828188101600896/t0(0) o400->MGC192.168.206.107@tcp@192.168.206.107@tcp:26/25 lens 224/224 e 0 to 1 dl 1743497769 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 [ 1913.733923] LustreError: MGC192.168.206.107@tcp: Connection to MGS (at 192.168.206.107@tcp) was lost; in progress operations using this service will fail [ 1938.407520] Lustre: Evicted from MGS (at 192.168.206.107@tcp) after server handle changed from 0x4e0cee60614ee967 to 0x4e0cee60614eea5c [ 1938.417963] Lustre: MGC192.168.206.107@tcp: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 1969.151935] LustreError: lustre-MDT0000-mdc-ffff926bc9bf4000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. [ 1969.179281] Lustre: lustre-MDT0000-mdc-ffff926bc9bf4000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 1979.808985] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1983.160658] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1989.425790] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1992.819690] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1996.351174] Lustre: Unmounted lustre-client [ 2049.258492] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 04:58:21 (1743497901) [ 2162.795470] Lustre: Mounted lustre-client [ 2170.368555] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2187.489464] Lustre: Unmounted lustre-client [ 2253.900629] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 05:01:47 (1743498107) [ 2258.162352] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 2261.659734] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 05:01:54 (1743498114) [ 2338.781884] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2364.404722] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2365.426794] Lustre: Mounted lustre-client [ 2365.428407] Lustre: Skipped 2 previous similar messages [ 2368.756724] Lustre: DEBUG MARKER: create test files [ 2397.315354] WARNING: CPU: 0 PID: 8 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 2397.323093] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 2397.352336] CPU: 0 PID: 8 Comm: kworker/u8:0 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #7 [ 2397.360859] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 2397.373929] Workqueue: writeback wb_workfn (flush-lustre-ffff926bc9) [ 2397.376934] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 2397.379612] Code: e8 b8 3b 1b f7 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 2397.397612] RSP: 0018:ffffa5ef80c97b58 EFLAGS: 00010202 [ 2397.399534] RAX: 0000000000000000 RBX: ffffa5ef80c97c40 RCX: 0000000000000001 [ 2397.411900] RDX: 0000000000000007 RSI: ffff926bd3dcc0c8 RDI: 0000000000000202 [ 2397.414534] RBP: ffff926bc8f4e310 R08: 0000000000000000 R09: 0000000000000001 [ 2397.417374] R10: 0000000007fff54c R11: ffff926bce0e0000 R12: ffff926bc8f4e550 [ 2397.420361] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 2397.436240] FS: 0000000000000000(0000) GS:ffff926c01200000(0000) knlGS:0000000000000000 [ 2397.442695] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2397.449273] CR2: 0000562d0fcdb028 CR3: 0000000116e28005 CR4: 0000000000170ef0 [ 2397.455962] Call Trace: [ 2397.456922] ? show_regs.cold.9+0x22/0x2f [ 2397.458606] ? __warn+0xc8/0x150 [ 2397.468491] ? ll_writepages+0x3dd/0x400 [lustre] [ 2397.474535] ? report_bug+0x113/0x140 [ 2397.479507] ? do_error_trap+0xb6/0x130 [ 2397.481239] ? do_invalid_op+0x46/0x60 [ 2397.482572] ? ll_writepages+0x3dd/0x400 [lustre] [ 2397.489345] ? invalid_op+0x14/0x20 [ 2397.491749] ? ll_writepages+0x3dd/0x400 [lustre] [ 2397.493488] ? ll_writepages+0x3b8/0x400 [lustre] [ 2397.495401] do_writepages+0x7a/0x270 [ 2397.496895] __writeback_single_inode+0xb2/0x7d0 [ 2397.502229] ? do_raw_spin_unlock+0x75/0x190 [ 2397.503384] writeback_sb_inodes+0x239/0x7e0 [ 2397.504424] wb_writeback+0x1ae/0x7a0 [ 2397.514074] wb_workfn+0x1e5/0x950 [ 2397.517333] ? lock_acquire+0x146/0x3a0 [ 2397.527945] ? process_one_work+0x2c0/0x9b0 [ 2397.529554] process_one_work+0x3dd/0x9b0 [ 2397.533789] worker_thread+0x296/0x6e0 [ 2397.537865] ? rescuer_thread+0x570/0x570 [ 2397.541854] kthread+0x1d7/0x210 [ 2397.542920] ? set_kthread_struct+0x70/0x70 [ 2397.544354] ret_from_fork+0x1f/0x30 [ 2397.549369] ---[ end trace cbd5b2721f509083 ]--- [ 2402.285643] Lustre: Unmounted lustre-client [ 2402.287334] Lustre: Skipped 2 previous similar messages [ 2426.954569] Key type lgssc unregistered [ 2427.626909] LNet: 32237:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2428.658685] LNet: Removed LNI 192.168.206.7@tcp [ 2430.707359] Key type .llcrypt unregistered [ 2430.709163] Key type ._llcrypt unregistered [ 2468.879469] Key type ._llcrypt registered [ 2468.881156] Key type .llcrypt registered [ 2470.241978] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2470.278185] alg: No test for adler32 (adler32-zlib) [ 2471.757091] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 2472.303469] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 2472.307355] LNet: Accept secure, port 988 [ 2474.224414] Key type lgssc registered [ 2476.791078] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2521.294383] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2544.685490] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2546.737060] Lustre: Mounted lustre-client [ 2552.446779] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 2560.300799] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 2564.773540] WARNING: CPU: 0 PID: 33836 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 2564.777350] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 2564.795924] CPU: 0 PID: 33836 Comm: kworker/u8:4 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 2564.800895] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 2564.804683] Workqueue: writeback wb_workfn (flush-lustre-ffff926c00) [ 2564.807428] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 2564.809869] Code: e8 b8 3b 1b f7 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 2564.816269] RSP: 0018:ffffa5ef88bffb58 EFLAGS: 00010202 [ 2564.818382] RAX: 0000000000000000 RBX: ffffa5ef88bffc40 RCX: 0000000000000001 [ 2564.821026] RDX: 0000000000000007 RSI: ffff926becd340c8 RDI: 0000000000000206 [ 2564.823287] RBP: ffff926bca2ed2d0 R08: 0000000000000000 R09: 0000000000000001 [ 2564.826539] R10: 000000000a61e3f1 R11: ffff926be4e1b000 R12: ffff926bca2ed510 [ 2564.829429] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 2564.831233] FS: 0000000000000000(0000) GS:ffff926c01200000(0000) knlGS:0000000000000000 [ 2564.833680] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2564.835936] CR2: 0000562934b50ae8 CR3: 0000000116e28006 CR4: 0000000000170ef0 [ 2564.838950] Call Trace: [ 2564.839731] ? show_regs.cold.9+0x22/0x2f [ 2564.841584] ? __warn+0xc8/0x150 [ 2564.842789] ? ll_writepages+0x3dd/0x400 [lustre] [ 2564.844958] ? report_bug+0x113/0x140 [ 2564.846489] ? do_error_trap+0xb6/0x130 [ 2564.848272] ? do_invalid_op+0x46/0x60 [ 2564.849840] ? ll_writepages+0x3dd/0x400 [lustre] [ 2564.851880] ? invalid_op+0x14/0x20 [ 2564.853236] ? ll_writepages+0x3dd/0x400 [lustre] [ 2564.855305] ? ll_writepages+0x3b8/0x400 [lustre] [ 2564.856902] do_writepages+0x7a/0x270 [ 2564.858335] __writeback_single_inode+0xb2/0x7d0 [ 2564.859864] ? do_raw_spin_unlock+0x75/0x190 [ 2564.861500] writeback_sb_inodes+0x239/0x7e0 [ 2564.862670] wb_writeback+0x1ae/0x7a0 [ 2564.863693] wb_workfn+0x1e5/0x950 [ 2564.865247] ? lock_acquire+0x146/0x3a0 [ 2564.866389] ? process_one_work+0x2c0/0x9b0 [ 2564.867746] process_one_work+0x3dd/0x9b0 [ 2564.869595] worker_thread+0x296/0x6e0 [ 2564.871159] ? rescuer_thread+0x570/0x570 [ 2564.872362] kthread+0x1d7/0x210 [ 2564.873387] ? set_kthread_struct+0x70/0x70 [ 2564.874906] ret_from_fork+0x1f/0x30 [ 2564.875903] ---[ end trace cbd5b2721f509084 ]--- [ 2565.891082] Lustre: Unmounted lustre-client [ 2584.254850] Key type lgssc unregistered [ 2584.903533] LNet: 34967:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2585.962390] LNet: Removed LNI 192.168.206.7@tcp [ 2587.742249] Key type .llcrypt unregistered [ 2587.744523] Key type ._llcrypt unregistered [ 2626.406898] Key type ._llcrypt registered [ 2626.408648] Key type .llcrypt registered [ 2627.136596] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2627.157518] alg: No test for adler32 (adler32-zlib) [ 2628.594951] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 2629.059955] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 2629.062745] LNet: Accept secure, port 988 [ 2630.920359] Key type lgssc registered [ 2633.289519] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2677.422385] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2700.641625] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2702.709638] Lustre: Mounted lustre-client [ 2708.235703] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2716.788742] Lustre: Unmounted lustre-client [ 2740.301060] Key type lgssc unregistered [ 2740.942368] LNet: 37588:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2741.996868] LNet: Removed LNI 192.168.206.7@tcp [ 2744.019288] Key type .llcrypt unregistered [ 2744.020923] Key type ._llcrypt unregistered [ 2786.869658] Key type ._llcrypt registered [ 2786.871507] Key type .llcrypt registered [ 2787.243730] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2787.265560] alg: No test for adler32 (adler32-zlib) [ 2788.674037] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 2789.133917] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 2789.139271] LNet: Accept secure, port 988 [ 2791.024479] Key type lgssc registered [ 2793.329389] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2895.357287] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2900.529429] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2923.997347] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2977.490480] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 05:13:51 (1743498831) [ 3133.292855] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3138.283530] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3160.518521] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3215.914578] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 05:17:49 (1743499069) [ 3251.145788] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3256.522436] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3277.705373] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3335.210417] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 3389.700313] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 05:20:43 (1743499243) [ 3392.010200] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 3394.937473] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 05:20:48 (1743499248) [ 3397.195755] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 3399.677349] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 05:20:53 (1743499253) [ 3401.772276] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 3404.162464] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 05:20:58 (1743499258) [ 3406.490964] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 3415.500674] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 3422.981275] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 3552.090033] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3556.359982] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3575.906290] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3624.368557] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 05:24:38 (1743499478) [ 3651.866242] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3656.154990] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3674.330827] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3702.072704] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 3711.346434] Lustre: Mounted lustre-client [ 3712.028527] Lustre: Mounted lustre-client [ 3727.263195] Lustre: DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 [ 3758.071143] Lustre: lustre-MDT0000-mdc-ffff926bed554000: Connection to lustre-MDT0000 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 3774.432198] Lustre: 38211:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743499614/real 1743499614] req@000000007a04e493 x1828190835835264/t0(0) o400->MGC192.168.206.107@tcp@192.168.206.107@tcp:26/25 lens 224/224 e 0 to 1 dl 1743499630 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 3774.443300] LustreError: MGC192.168.206.107@tcp: Connection to MGS (at 192.168.206.107@tcp) was lost; in progress operations using this service will fail [ 3784.691493] Lustre: Evicted from MGS (at 192.168.206.107@tcp) after server handle changed from 0x2856c61785023d95 to 0x2856c6178502fe4d [ 3784.704303] Lustre: MGC192.168.206.107@tcp: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 3792.448287] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@000000004ca692c6 x1828190835191808/t8589934597(8589934597) o101->lustre-MDT0000-mdc-ffff926bed554000@192.168.206.107@tcp:12/10 lens 592/608 e 0 to 0 dl 1743499664 ref 2 fl Interpret:RQU/204/0 rc 301/301 job:'' uid:0 gid:0 [ 3793.105615] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@00000000ef42d5d3 x1828190835192448/t8589934599(8589934599) o101->lustre-MDT0000-mdc-ffff926bed554000@192.168.206.107@tcp:12/10 lens 592/608 e 0 to 0 dl 1743499664 ref 2 fl Interpret:RQU/204/0 rc 301/301 job:'' uid:0 gid:0 [ 3794.432859] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@00000000a37fca19 x1828190835193728/t8589934603(8589934603) o101->lustre-MDT0000-mdc-ffff926bed554000@192.168.206.107@tcp:12/10 lens 592/608 e 0 to 0 dl 1743499666 ref 2 fl Interpret:RQU/204/0 rc 301/301 job:'' uid:0 gid:0 [ 3794.444330] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) Skipped 1 previous similar message [ 3797.022848] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@0000000060df9391 x1828190835196288/t8589934611(8589934611) o101->lustre-MDT0000-mdc-ffff926bed554000@192.168.206.107@tcp:12/10 lens 592/608 e 0 to 0 dl 1743499668 ref 2 fl Interpret:RQU/204/0 rc 301/301 job:'' uid:0 gid:0 [ 3797.033060] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) Skipped 3 previous similar messages [ 3801.501803] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@00000000af677d53 x1828190835200768/t8589934625(8589934625) o101->lustre-MDT0000-mdc-ffff926bed554000@192.168.206.107@tcp:12/10 lens 592/608 e 0 to 0 dl 1743499673 ref 2 fl Interpret:RQU/204/0 rc 301/301 job:'' uid:0 gid:0 [ 3801.528456] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) Skipped 6 previous similar messages [ 3809.795716] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@00000000b521fc62 x1828190835209088/t8589934651(8589934651) o101->lustre-MDT0000-mdc-ffff926bed554000@192.168.206.107@tcp:12/10 lens 592/608 e 0 to 0 dl 1743499681 ref 2 fl Interpret:RQU/204/0 rc 301/301 job:'' uid:0 gid:0 [ 3809.807635] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) Skipped 12 previous similar messages [ 3819.266790] LustreError: 38209:0:(mdc_request.c:660:mdc_replay_open()) @@@ cannot properly replay without open data req@00000000854b7bd0 x1828190835218688/t8589934681(8589934681) o101->lustre-MDT0000-mdc-ffff926bed554000@192.168.206.107@tcp:12/10 lens 592/608 e 0 to 0 dl 1743499691 ref 1 fl Interpret:RMQU/204/0 rc 301/301 job:'' uid:0 gid:0 [ 3826.296384] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@0000000051b48739 x1828190835225728/t8589934703(8589934703) o101->lustre-MDT0000-mdc-ffff926bed554000@192.168.206.107@tcp:12/10 lens 592/608 e 0 to 0 dl 1743499698 ref 2 fl Interpret:RQU/204/0 rc 301/301 job:'' uid:0 gid:0 [ 3826.306850] LustreError: 38209:0:(client.c:3268:ptlrpc_replay_interpret()) Skipped 25 previous similar messages [ 3849.756824] LustreError: lustre-MDT0000-mdc-ffff926bed554000: operation ldlm_enqueue to node 192.168.206.107@tcp failed: rc = -107 [ 3849.822210] Lustre: lustre-MDT0000-mdc-ffff926bf90c4000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 3854.319288] LustreError: lustre-MDT0000-mdc-ffff926bed554000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. [ 3854.549445] LustreError: 58352:0:(file.c:6007:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -5 [ 3855.119422] Lustre: lustre-MDT0000-mdc-ffff926bed554000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 3861.502987] Lustre: Unmounted lustre-client [ 3863.015589] Lustre: Unmounted lustre-client [ 3914.482621] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 05:29:28 (1743499768) [ 3986.804633] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4000.760166] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 05:30:54 (1743499854) [ 4035.208971] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4158.822969] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4163.460242] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4185.388440] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4236.159991] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 05:34:50 (1743500090) [ 4242.145567] Key type lgssc unregistered [ 4242.717282] LNet: 65470:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4243.752460] LNet: Removed LNI 192.168.206.7@tcp [ 4245.197247] Key type .llcrypt unregistered [ 4245.202419] Key type ._llcrypt unregistered [ 4279.498909] Key type ._llcrypt registered [ 4279.500565] Key type .llcrypt registered [ 4279.842223] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 4279.867293] alg: No test for adler32 (adler32-zlib) [ 4281.158926] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 4281.493761] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 4281.496904] LNet: Accept secure, port 988 [ 4283.257205] Key type lgssc registered [ 4285.158479] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4393.074969] Lustre: Mounted lustre-client [ 4394.412183] WARNING: CPU: 0 PID: 50185 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 4394.416455] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 4394.435459] CPU: 0 PID: 50185 Comm: kworker/u8:1 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 4394.439395] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 4394.443007] Workqueue: writeback wb_workfn (flush-lustre-ffff926bed) [ 4394.445507] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 4394.447332] Code: e8 b8 3b 1b f7 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 4394.453440] RSP: 0018:ffffa5ef88abfb58 EFLAGS: 00010202 [ 4394.455437] RAX: 0000000000000000 RBX: ffffa5ef88abfc40 RCX: 0000000000000001 [ 4394.458303] RDX: 0000000000000007 RSI: ffff926bdfb090c8 RDI: 0000000000000206 [ 4394.461031] RBP: ffff926bf975b250 R08: 0000000000000000 R09: 0000000000000001 [ 4394.463749] R10: 000000000c2aed78 R11: ffff926bf924a000 R12: ffff926bf975b490 [ 4394.465784] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 4394.468116] FS: 0000000000000000(0000) GS:ffff926c01200000(0000) knlGS:0000000000000000 [ 4394.471188] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4394.473331] CR2: 000055b076677730 CR3: 0000000116e28003 CR4: 0000000000170ef0 [ 4394.475756] Call Trace: [ 4394.476498] ? show_regs.cold.9+0x22/0x2f [ 4394.477934] ? __warn+0xc8/0x150 [ 4394.479147] ? ll_writepages+0x3dd/0x400 [lustre] [ 4394.480846] ? report_bug+0x113/0x140 [ 4394.482425] ? do_error_trap+0xb6/0x130 [ 4394.483803] ? do_invalid_op+0x46/0x60 [ 4394.485310] ? ll_writepages+0x3dd/0x400 [lustre] [ 4394.487290] ? invalid_op+0x14/0x20 [ 4394.488578] ? ll_writepages+0x3dd/0x400 [lustre] [ 4394.490489] ? ll_writepages+0x3b8/0x400 [lustre] [ 4394.492375] do_writepages+0x7a/0x270 [ 4394.493821] __writeback_single_inode+0xb2/0x7d0 [ 4394.495448] ? do_raw_spin_unlock+0x75/0x190 [ 4394.496685] writeback_sb_inodes+0x239/0x7e0 [ 4394.498310] wb_writeback+0x1ae/0x7a0 [ 4394.499842] wb_workfn+0x1e5/0x950 [ 4394.501274] ? lock_acquire+0x146/0x3a0 [ 4394.502794] ? process_one_work+0x2c0/0x9b0 [ 4394.504462] process_one_work+0x3dd/0x9b0 [ 4394.506764] worker_thread+0x296/0x6e0 [ 4394.508208] ? rescuer_thread+0x570/0x570 [ 4394.509778] kthread+0x1d7/0x210 [ 4394.510973] ? set_kthread_struct+0x70/0x70 [ 4394.512512] ret_from_fork+0x1f/0x30 [ 4394.513778] ---[ end trace cbd5b2721f509085 ]--- [ 4394.618689] Lustre: Unmounted lustre-client [ 4428.284394] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 05:38:02 (1743500282) [ 4470.908079] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 05:38:44 (1743500324) [ 4586.205076] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4590.351957] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4610.115917] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4650.173352] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 05:41:44 (1743500504) [ 4677.045685] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4681.028482] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4697.401490] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4703.269924] Lustre: Mounted lustre-client [ 4723.680202] Lustre: 76498:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743500563/real 1743500563] req@000000003145e9d1 x1828192399682176/t0(0) o36->lustre-MDT0000-mdc-ffff926bf9254000@192.168.206.107@tcp:12/10 lens 488/512 e 0 to 1 dl 1743500579 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 4723.680713] Lustre: lustre-MDT0000-mdc-ffff926bf9254000: Connection to lustre-MDT0000 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 4723.703691] Lustre: 76498:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 2 previous similar messages [ 4723.729674] Lustre: lustre-MDT0000-mdc-ffff926bf9254000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 4741.600257] Lustre: 76539:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743500581/real 1743500581] req@0000000060de92b4 x1828192399691904/t0(0) o36->lustre-MDT0000-mdc-ffff926bf9254000@192.168.206.107@tcp:12/10 lens 488/512 e 0 to 1 dl 1743500597 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 4741.600856] Lustre: lustre-MDT0000-mdc-ffff926bf9254000: Connection to lustre-MDT0000 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 4741.621679] Lustre: 76539:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [ 4741.670422] Lustre: lustre-MDT0000-mdc-ffff926bf9254000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 4743.292220] Lustre: Unmounted lustre-client [ 4773.163271] Key type lgssc unregistered [ 4773.679225] LNet: 76979:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4774.697120] LNet: Removed LNI 192.168.206.7@tcp [ 4776.096134] Key type .llcrypt unregistered [ 4776.097938] Key type ._llcrypt unregistered [ 4798.015361] Key type ._llcrypt registered [ 4798.017034] Key type .llcrypt registered [ 4799.820497] Lustre: DEBUG MARKER: == conf-sanity test 90b: check max_mod_rpcs_in_flight is enforced after update ========================================================== 05:44:13 (1743500653) [ 4803.572674] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4803.595271] alg: No test for adler32 (adler32-zlib) [ 4804.852597] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 4805.201614] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 4805.205881] LNet: Accept secure, port 988 [ 4806.976404] Key type lgssc registered [ 4808.756205] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4854.663190] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4858.956791] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4875.965903] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4881.951864] Lustre: Mounted lustre-client [ 4905.440237] Lustre: 79943:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743500745/real 1743500745] req@00000000fbd4098e x1828192949118208/t0(0) o36->lustre-MDT0000-mdc-ffff926be50e1000@192.168.206.107@tcp:12/10 lens 488/512 e 0 to 1 dl 1743500761 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 4905.453496] Lustre: lustre-MDT0000-mdc-ffff926be50e1000: Connection to lustre-MDT0000 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 4905.484763] Lustre: lustre-MDT0000-mdc-ffff926be50e1000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 4915.683882] WARNING: CPU: 1 PID: 50185 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 4915.688627] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 4915.704042] CPU: 1 PID: 50185 Comm: kworker/u8:1 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 4915.707977] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 4915.710897] Workqueue: writeback wb_workfn (flush-lustre-ffff926be5) [ 4915.713248] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 4915.715127] Code: e8 b8 3b 1b f7 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 4915.720749] RSP: 0018:ffffa5ef88abfb18 EFLAGS: 00010202 [ 4915.722554] RAX: 0000000000000000 RBX: ffffa5ef88abfc00 RCX: 0000000000000001 [ 4915.725196] RDX: 000000000000000e RSI: ffff926bec9090c8 RDI: 0000000000000206 [ 4915.727774] RBP: ffff926bf975c290 R08: 0000000000000000 R09: 0000000000000001 [ 4915.729697] R10: 000000000c882038 R11: ffff926bed6d2000 R12: ffff926bf975c4d0 [ 4915.732145] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000 [ 4915.734654] FS: 0000000000000000(0000) GS:ffff926c01600000(0000) knlGS:0000000000000000 [ 4915.737377] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4915.739708] CR2: 00007f0467810070 CR3: 0000000116e28004 CR4: 0000000000170ee0 [ 4915.742186] Call Trace: [ 4915.743325] ? show_regs.cold.9+0x22/0x2f [ 4915.744851] ? __warn+0xc8/0x150 [ 4915.745938] ? ll_writepages+0x3dd/0x400 [lustre] [ 4915.747835] ? report_bug+0x113/0x140 [ 4915.748908] ? do_error_trap+0xb6/0x130 [ 4915.750022] ? do_invalid_op+0x46/0x60 [ 4915.751126] ? ll_writepages+0x3dd/0x400 [lustre] [ 4915.752732] ? invalid_op+0x14/0x20 [ 4915.753861] ? ll_writepages+0x3dd/0x400 [lustre] [ 4915.755514] ? ll_writepages+0x3b8/0x400 [lustre] [ 4915.757036] do_writepages+0x7a/0x270 [ 4915.758209] __writeback_single_inode+0xb2/0x7d0 [ 4915.759832] ? do_raw_spin_unlock+0x75/0x190 [ 4915.761360] writeback_sb_inodes+0x239/0x7e0 [ 4915.762907] __writeback_inodes_wb+0x71/0x110 [ 4915.764306] wb_writeback+0x627/0x7a0 [ 4915.765606] ? get_nr_inodes+0x45/0x70 [ 4915.766890] wb_workfn+0x778/0x950 [ 4915.767766] ? sched_clock_cpu+0x24/0x150 [ 4915.769044] process_one_work+0x3dd/0x9b0 [ 4915.770458] worker_thread+0x296/0x6e0 [ 4915.771677] ? rescuer_thread+0x570/0x570 [ 4915.772646] kthread+0x1d7/0x210 [ 4915.773398] ? set_kthread_struct+0x70/0x70 [ 4915.774684] ret_from_fork+0x1f/0x30 [ 4915.775611] ---[ end trace cbd5b2721f509086 ]--- [ 4923.360261] Lustre: 79989:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743500763/real 1743500763] req@0000000069c35349 x1828192949131648/t0(0) o36->lustre-MDT0001-mdc-ffff926be50e1000@192.168.206.107@tcp:12/10 lens 488/512 e 0 to 1 dl 1743500779 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 4923.361857] Lustre: lustre-MDT0001-mdc-ffff926be50e1000: Connection to lustre-MDT0001 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 4923.386057] Lustre: 79989:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [ 4923.410154] Lustre: lustre-MDT0001-mdc-ffff926be50e1000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 4941.280244] Lustre: 80020:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743500781/real 1743500781] req@00000000ec1de0fe x1828192949137280/t0(0) o36->lustre-MDT0001-mdc-ffff926be50e1000@192.168.206.107@tcp:12/10 lens 488/512 e 0 to 1 dl 1743500797 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 4941.280469] Lustre: lustre-MDT0001-mdc-ffff926be50e1000: Connection to lustre-MDT0001 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 4941.294048] Lustre: 80020:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [ 4941.325378] Lustre: lustre-MDT0001-mdc-ffff926be50e1000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 4944.376092] Lustre: Unmounted lustre-client [ 4945.633325] Lustre: Mounted lustre-client [ 4964.320246] Lustre: 80171:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743500804/real 1743500804] req@00000000d1978e26 x1828192949173632/t0(0) o36->lustre-MDT0000-mdc-ffff926bdfb54000@192.168.206.107@tcp:12/10 lens 488/512 e 0 to 1 dl 1743500820 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 4964.320464] Lustre: lustre-MDT0000-mdc-ffff926bdfb54000: Connection to lustre-MDT0000 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 4964.335255] Lustre: 80171:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [ 4964.355697] Lustre: lustre-MDT0000-mdc-ffff926bdfb54000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 4981.218292] Lustre: 80219:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743500821/real 1743500821] req@00000000a7c415ff x1828192949188992/t0(0) o36->lustre-MDT0000-mdc-ffff926bdfb54000@192.168.206.107@tcp:12/10 lens 488/512 e 0 to 1 dl 1743500837 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 4981.218580] Lustre: lustre-MDT0000-mdc-ffff926bdfb54000: Connection to lustre-MDT0000 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 4981.232848] Lustre: 80219:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 4981.266615] Lustre: lustre-MDT0000-mdc-ffff926bdfb54000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 4985.211085] Lustre: Unmounted lustre-client [ 5013.997443] Key type lgssc unregistered [ 5014.487367] LNet: 80693:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5015.527376] LNet: Removed LNI 192.168.206.7@tcp [ 5016.872487] Key type .llcrypt unregistered [ 5016.874068] Key type ._llcrypt unregistered [ 5037.386151] Key type ._llcrypt registered [ 5037.387661] Key type .llcrypt registered [ 5039.307906] Lustre: DEBUG MARKER: == conf-sanity test 90c: check max_mod_rpcs_in_flight update limits ========================================================== 05:48:13 (1743500893) [ 5043.030143] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5043.051656] alg: No test for adler32 (adler32-zlib) [ 5044.258270] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 5044.600664] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 5044.603645] LNet: Accept secure, port 988 [ 5046.360399] Key type lgssc registered [ 5048.362738] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5094.360452] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5098.830821] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5117.488813] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5123.612019] Lustre: Mounted lustre-client [ 5129.384351] Lustre: Unmounted lustre-client [ 5130.855445] Lustre: Mounted lustre-client [ 5131.266549] LustreError: 83725:0:(genops.c:2143:obd_set_max_mod_rpcs_in_flight()) lustre-MDT0000-mdc-ffff926bee831000: can't set max_mod_rpcs_in_flight=9 higher than mdt.*.max_mod_rpcs_in_flight=8 returned by the MDT server at connection. [ 5134.449891] Lustre: Unmounted lustre-client [ 5135.062275] Lustre: Mounted lustre-client [ 5139.822941] Lustre: Unmounted lustre-client [ 5168.992522] Key type lgssc unregistered [ 5169.441650] LNet: 84273:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5170.483628] LNet: Removed LNI 192.168.206.7@tcp [ 5171.813976] Key type .llcrypt unregistered [ 5171.815660] Key type ._llcrypt unregistered [ 5192.769345] Key type ._llcrypt registered [ 5192.772493] Key type .llcrypt registered [ 5194.807138] Lustre: DEBUG MARKER: == conf-sanity test 90d: check one close RPC is allowed above max_mod_rpcs_in_flight ========================================================== 05:50:48 (1743501048) [ 5198.639827] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5198.654375] alg: No test for adler32 (adler32-zlib) [ 5199.945620] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 5200.357118] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 5200.360598] LNet: Accept secure, port 988 [ 5202.168435] Key type lgssc registered [ 5204.029615] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5249.288327] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5253.877601] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5272.214821] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5273.918569] Lustre: Mounted lustre-client [ 5294.560203] Lustre: 87231:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743501134/real 1743501134] req@000000006eb03523 x1828193363313792/t0(0) o36->lustre-MDT0000-mdc-ffff926bed319000@192.168.206.107@tcp:12/10 lens 488/512 e 0 to 1 dl 1743501150 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 5294.560472] Lustre: lustre-MDT0000-mdc-ffff926bed319000: Connection to lustre-MDT0000 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 5294.579682] Lustre: 87231:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [ 5294.606168] Lustre: lustre-MDT0000-mdc-ffff926bed319000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [ 5308.717794] Lustre: Unmounted lustre-client [ 5335.929170] Key type lgssc unregistered [ 5336.380259] LNet: 87678:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5337.453443] LNet: Removed LNI 192.168.206.7@tcp [ 5338.700610] Key type .llcrypt unregistered [ 5338.702298] Key type ._llcrypt unregistered [ 5358.091469] Key type ._llcrypt registered [ 5358.092993] Key type .llcrypt registered [ 5359.868867] Lustre: DEBUG MARKER: == conf-sanity test 91: evict-by-nid support ============= 05:53:33 (1743501213) [ 5363.941920] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5363.960840] alg: No test for adler32 (adler32-zlib) [ 5365.154073] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 5365.499677] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 5365.503813] LNet: Accept secure, port 988 [ 5367.256366] Key type lgssc registered [ 5369.084226] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5409.671198] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5413.429733] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5429.029625] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5440.006988] Lustre: Mounted lustre-client [ 5445.060536] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 5455.426137] Lustre: Unmounted lustre-client [ 5509.957596] Key type lgssc unregistered [ 5510.409759] LNet: 91137:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5511.463293] LNet: Removed LNI 192.168.206.7@tcp [ 5512.658515] Key type .llcrypt unregistered [ 5512.660325] Key type ._llcrypt unregistered [ 5533.149181] Key type ._llcrypt registered [ 5533.150793] Key type .llcrypt registered [ 5535.178779] Lustre: DEBUG MARKER: == conf-sanity test 92: ldev returns MGS NID correctly in command substitution ========================================================== 05:56:28 (1743501388) [ 5548.460841] Lustre: DEBUG MARKER: == conf-sanity test 93: register mulitple MDT at the same time ========================================================== 05:56:42 (1743501402) [ 5550.983785] Lustre: DEBUG MARKER: SKIP: conf-sanity test_93 needs >= 3 MDTs [ 5553.473957] Lustre: DEBUG MARKER: == conf-sanity test 94: ldev outputs correct labels for file system name query ========================================================== 05:56:47 (1743501407) [ 5564.940670] Lustre: DEBUG MARKER: == conf-sanity test 95: ldev should only allow one label filter ========================================================== 05:56:59 (1743501419) [ 5576.923647] Lustre: DEBUG MARKER: == conf-sanity test 96: ldev returns hostname and backend fs correctly in command sub ========================================================== 05:57:11 (1743501431) [ 5589.220769] Lustre: DEBUG MARKER: == conf-sanity test 97: ldev returns correct ouput when querying based on role ========================================================== 05:57:23 (1743501443) [ 5601.077083] Lustre: DEBUG MARKER: == conf-sanity test 98: Buffer-overflow check while parsing mount_opts ========================================================== 05:57:35 (1743501455) [ 5604.594651] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5604.616444] alg: No test for adler32 (adler32-zlib) [ 5605.836552] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 5606.176306] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 5606.179230] LNet: Accept secure, port 988 [ 5607.928293] Key type lgssc registered [ 5609.636095] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5650.028947] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5653.499909] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5667.076789] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5678.081158] Lustre: Mounted lustre-client [ 5682.161438] Lustre: Unmounted lustre-client [ 5707.894876] Key type lgssc unregistered [ 5708.340161] LNet: 98716:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5709.415814] LNet: Removed LNI 192.168.206.7@tcp [ 5710.651062] Key type .llcrypt unregistered [ 5710.652248] Key type ._llcrypt unregistered [ 5728.235285] Key type ._llcrypt registered [ 5728.236568] Key type .llcrypt registered [ 5729.860379] Lustre: DEBUG MARKER: == conf-sanity test 99: Adding meta_bg option ============ 05:59:44 (1743501584) [ 5733.222200] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5733.246094] alg: No test for adler32 (adler32-zlib) [ 5734.406393] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 5734.704204] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 5734.706472] LNet: Accept secure, port 988 [ 5736.432375] Key type lgssc registered [ 5738.146686] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5766.000701] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 06:00:20 (1743501620) [ 5834.588350] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5838.085467] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5852.503423] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5854.095777] Lustre: Mounted lustre-client [ 5857.284770] Lustre: Unmounted lustre-client [ 5881.760118] Key type lgssc unregistered [ 5882.180916] LNet: 104664:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5883.239299] LNet: Removed LNI 192.168.206.7@tcp [ 5884.323460] Key type .llcrypt unregistered [ 5884.325034] Key type ._llcrypt unregistered [ 5901.727315] Key type ._llcrypt registered [ 5901.728966] Key type .llcrypt registered [ 5903.253664] Lustre: DEBUG MARKER: == conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 06:02:37 (1743501757) [ 5906.377796] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5906.400721] alg: No test for adler32 (adler32-zlib) [ 5907.578445] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 5907.860164] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 5907.862603] LNet: Accept secure, port 988 [ 5909.592271] Key type lgssc registered [ 5911.246925] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5948.347400] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5951.540922] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5964.072166] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5965.637655] Lustre: Mounted lustre-client [ 6972.315965] Lustre: Unmounted lustre-client [ 7013.359061] Key type lgssc unregistered [ 7013.764900] LNet: 116793:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7014.823576] LNet: Removed LNI 192.168.206.7@tcp [ 7016.115027] Key type .llcrypt unregistered [ 7016.116680] Key type ._llcrypt unregistered [ 7032.268897] Key type ._llcrypt registered [ 7032.270324] Key type .llcrypt registered [ 7033.843669] Lustre: DEBUG MARKER: == conf-sanity test 101b: Race events DISCONNECT and ACTIVE in osp ========================================================== 06:21:28 (1743502888) [ 7036.984158] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7037.006147] alg: No test for adler32 (adler32-zlib) [ 7038.226399] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 7038.526221] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 7038.528227] LNet: Accept secure, port 988 [ 7040.256365] Key type lgssc registered [ 7042.000093] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7075.893871] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7079.453022] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7093.330149] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7094.896539] Lustre: Mounted lustre-client [ 7100.390417] Lustre: lustre-OST0000-osc-ffff926bd0a51000: Connection to lustre-OST0000 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 7144.636704] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7157.507962] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 10 sec [ 7161.152276] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff926bd0a51000.ost_server_uuid 50 [ 7163.013525] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff926bd0a51000.ost_server_uuid in FULL state after 0 sec [ 7164.134613] WARNING: CPU: 3 PID: 8 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 7164.137057] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 7164.150794] CPU: 3 PID: 8 Comm: kworker/u8:0 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 7164.153922] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 7164.155588] Workqueue: writeback wb_workfn (flush-lustre-ffff926bd0) [ 7164.157180] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 7164.158379] Code: e8 b8 3b 1b f7 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 7164.162750] RSP: 0018:ffffa5ef80c97b58 EFLAGS: 00010202 [ 7164.163899] RAX: 0000000000000000 RBX: ffffa5ef80c97c40 RCX: 0000000000000001 [ 7164.166301] RDX: 0000000000000007 RSI: ffff926bfddc10c8 RDI: 0000000000000206 [ 7164.168420] RBP: ffff926bedcb6310 R08: 0000000000000000 R09: 0000000000000001 [ 7164.170562] R10: 000000000da9701a R11: ffff926beeb4b000 R12: ffff926bedcb6550 [ 7164.172875] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 7164.174705] FS: 0000000000000000(0000) GS:ffff926c01e00000(0000) knlGS:0000000000000000 [ 7164.177123] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 7164.178503] CR2: 000055e0dc10df28 CR3: 0000000116e28003 CR4: 0000000000170ee0 [ 7164.180140] Call Trace: [ 7164.180775] ? show_regs.cold.9+0x22/0x2f [ 7164.182219] ? __warn+0xc8/0x150 [ 7164.183481] ? ll_writepages+0x3dd/0x400 [lustre] [ 7164.184805] ? report_bug+0x113/0x140 [ 7164.185988] ? do_error_trap+0xb6/0x130 [ 7164.187161] ? do_invalid_op+0x46/0x60 [ 7164.188198] ? ll_writepages+0x3dd/0x400 [lustre] [ 7164.189418] ? invalid_op+0x14/0x20 [ 7164.190148] ? ll_writepages+0x3dd/0x400 [lustre] [ 7164.192007] ? ll_writepages+0x3b8/0x400 [lustre] [ 7164.193533] do_writepages+0x7a/0x270 [ 7164.194906] __writeback_single_inode+0xb2/0x7d0 [ 7164.196367] ? do_raw_spin_unlock+0x75/0x190 [ 7164.197392] writeback_sb_inodes+0x239/0x7e0 [ 7164.198195] wb_writeback+0x1ae/0x7a0 [ 7164.199207] wb_workfn+0x1e5/0x950 [ 7164.200515] ? lock_acquire+0x146/0x3a0 [ 7164.201897] ? process_one_work+0x2c0/0x9b0 [ 7164.203369] process_one_work+0x3dd/0x9b0 [ 7164.204805] worker_thread+0x296/0x6e0 [ 7164.206055] ? rescuer_thread+0x570/0x570 [ 7164.207449] kthread+0x1d7/0x210 [ 7164.208400] ? set_kthread_struct+0x70/0x70 [ 7164.209705] ret_from_fork+0x1f/0x30 [ 7164.210981] ---[ end trace cbd5b2721f509087 ]--- [ 7164.286746] Lustre: Unmounted lustre-client [ 7188.701943] Key type lgssc unregistered [ 7189.207737] LNet: 121241:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7190.249872] LNet: Removed LNI 192.168.206.7@tcp [ 7191.288377] Key type .llcrypt unregistered [ 7191.289333] Key type ._llcrypt unregistered [ 7207.901943] Key type ._llcrypt registered [ 7207.903057] Key type .llcrypt registered [ 7209.403371] Lustre: DEBUG MARKER: SKIP: conf-sanity test_102 skipping excluded test 102 [ 7211.169962] Lustre: DEBUG MARKER: == conf-sanity test 103: rename filesystem name ========== 06:24:25 (1743503065) [ 7219.866705] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7219.886303] alg: No test for adler32 (adler32-zlib) [ 7221.180185] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 7221.560173] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 7221.561971] LNet: Accept secure, port 988 [ 7223.280374] Key type lgssc registered [ 7225.031926] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7278.943136] Lustre: Mounted lustre-client [ 7284.112982] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7301.088375] WARNING: CPU: 3 PID: 8 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 7301.091066] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 7301.106617] CPU: 3 PID: 8 Comm: kworker/u8:0 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 7301.110099] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 7301.113133] Workqueue: writeback wb_workfn (flush-lustre-ffff926c00) [ 7301.115400] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 7301.117435] Code: e8 b8 3b 1b f7 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 7301.123892] RSP: 0018:ffffa5ef80c97b18 EFLAGS: 00010202 [ 7301.125765] RAX: 0000000000000000 RBX: ffffa5ef80c97c00 RCX: 0000000000000001 [ 7301.128191] RDX: 000000000000000e RSI: ffff926bfb3b90c8 RDI: 0000000000000202 [ 7301.130596] RBP: ffff926bedc0c290 R08: 0000000000000000 R09: 0000000000000001 [ 7301.132309] R10: 000000000dca9fc9 R11: ffff926becd14000 R12: ffff926bedc0c4d0 [ 7301.133957] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000 [ 7301.135490] FS: 0000000000000000(0000) GS:ffff926c01e00000(0000) knlGS:0000000000000000 [ 7301.138286] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 7301.140525] CR2: 000055eef4cae518 CR3: 0000000116e28001 CR4: 0000000000170ee0 [ 7301.142962] Call Trace: [ 7301.143816] ? show_regs.cold.9+0x22/0x2f [ 7301.145426] ? __warn+0xc8/0x150 [ 7301.146727] ? ll_writepages+0x3dd/0x400 [lustre] [ 7301.148704] ? report_bug+0x113/0x140 [ 7301.150010] ? do_error_trap+0xb6/0x130 [ 7301.151194] ? do_invalid_op+0x46/0x60 [ 7301.152142] ? ll_writepages+0x3dd/0x400 [lustre] [ 7301.153480] ? invalid_op+0x14/0x20 [ 7301.154373] ? ll_writepages+0x3dd/0x400 [lustre] [ 7301.155669] ? ll_writepages+0x3b8/0x400 [lustre] [ 7301.156965] do_writepages+0x7a/0x270 [ 7301.157908] __writeback_single_inode+0xb2/0x7d0 [ 7301.159135] ? do_raw_spin_unlock+0x75/0x190 [ 7301.160065] writeback_sb_inodes+0x239/0x7e0 [ 7301.161010] __writeback_inodes_wb+0x71/0x110 [ 7301.162465] wb_writeback+0x627/0x7a0 [ 7301.163812] wb_workfn+0x709/0x950 [ 7301.165070] ? sched_clock_cpu+0x24/0x150 [ 7301.166436] process_one_work+0x3dd/0x9b0 [ 7301.167583] worker_thread+0x296/0x6e0 [ 7301.168831] ? rescuer_thread+0x570/0x570 [ 7301.170035] kthread+0x1d7/0x210 [ 7301.171177] ? set_kthread_struct+0x70/0x70 [ 7301.172579] ret_from_fork+0x1f/0x30 [ 7301.173737] ---[ end trace cbd5b2721f509088 ]--- [ 7303.167869] Lustre: Unmounted lustre-client [ 7395.694669] Lustre: Mounted mylustre-client [ 7401.265812] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7414.719934] Lustre: Unmounted mylustre-client [ 7484.568795] Lustre: Mounted tfs-client [ 7489.887647] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7500.867384] Lustre: Unmounted tfs-client [ 7577.264824] Lustre: Mounted lustre-client [ 7582.497806] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7597.571625] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 06:30:52 (1743503452) [ 7598.871985] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 7609.181924] Lustre: Unmounted lustre-client [ 7737.381829] Lustre: client wants to enable acl, but mdt not! [ 7737.433793] Lustre: Mounted lustre-client [ 7746.053642] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 06:33:20 (1743503600) [ 7746.674101] Lustre: client wants to enable acl, but mdt not! [ 7746.717515] Lustre: Mounted lustre-client [ 7748.314181] Lustre: Unmounted lustre-client [ 7748.765788] Lustre: client wants to enable acl, but mdt not! [ 7757.832481] Lustre: Unmounted lustre-client [ 7757.833917] Lustre: Skipped 1 previous similar message [ 7759.897101] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 06:33:34 (1743503614) [ 7761.113796] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 7761.115622] Lustre: Skipped 3 previous similar messages [ 7803.048557] Key type lgssc unregistered [ 7803.545460] LNet: 136017:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7804.583083] LNet: Removed LNI 192.168.206.7@tcp [ 7805.700565] Key type .llcrypt unregistered [ 7805.701867] Key type ._llcrypt unregistered [ 7825.422360] Key type ._llcrypt registered [ 7825.424501] Key type .llcrypt registered [ 7825.953658] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 7825.972385] alg: No test for adler32 (adler32-zlib) [ 7827.175419] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 7827.477786] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 7827.480284] LNet: Accept secure, port 988 [ 7829.216343] Key type lgssc registered [ 7830.881944] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7884.134243] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7887.512532] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7900.804746] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7902.660994] Lustre: Mounted lustre-client [ 7903.069305] Lustre: Remounted lustre-client read-only [ 7904.215434] WARNING: CPU: 0 PID: 131018 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 7904.217683] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 7904.229566] CPU: 0 PID: 131018 Comm: kworker/u8:2 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 7904.231666] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 7904.233632] Workqueue: writeback wb_workfn (flush-lustre-ffff926be6) [ 7904.234800] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 7904.235868] Code: e8 b8 3b 1b f7 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 7904.239397] RSP: 0018:ffffa5ef816bbb58 EFLAGS: 00010202 [ 7904.240363] RAX: 0000000000000000 RBX: ffffa5ef816bbc40 RCX: 0000000000000001 [ 7904.241698] RDX: 0000000000000007 RSI: ffff926bececc0c8 RDI: 0000000000000206 [ 7904.242978] RBP: ffff926bedcae310 R08: 0000000000000000 R09: 0000000000000001 [ 7904.244285] R10: 000000000e28771a R11: ffff926be5721000 R12: ffff926bedcae550 [ 7904.246940] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 7904.248797] FS: 0000000000000000(0000) GS:ffff926c01200000(0000) knlGS:0000000000000000 [ 7904.250255] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 7904.251312] CR2: 000055b076681f44 CR3: 0000000116e28006 CR4: 0000000000170ef0 [ 7904.252720] Call Trace: [ 7904.253179] ? show_regs.cold.9+0x22/0x2f [ 7904.253973] ? __warn+0xc8/0x150 [ 7904.254550] ? ll_writepages+0x3dd/0x400 [lustre] [ 7904.255524] ? report_bug+0x113/0x140 [ 7904.256245] ? do_error_trap+0xb6/0x130 [ 7904.257373] ? do_invalid_op+0x46/0x60 [ 7904.258600] ? ll_writepages+0x3dd/0x400 [lustre] [ 7904.260259] ? invalid_op+0x14/0x20 [ 7904.261411] ? ll_writepages+0x3dd/0x400 [lustre] [ 7904.263037] ? ll_writepages+0x3b8/0x400 [lustre] [ 7904.264524] do_writepages+0x7a/0x270 [ 7904.265708] __writeback_single_inode+0xb2/0x7d0 [ 7904.267142] ? do_raw_spin_unlock+0x75/0x190 [ 7904.268564] writeback_sb_inodes+0x239/0x7e0 [ 7904.269957] wb_writeback+0x1ae/0x7a0 [ 7904.271152] wb_workfn+0x1e5/0x950 [ 7904.272242] ? lock_acquire+0x146/0x3a0 [ 7904.273505] ? process_one_work+0x2c0/0x9b0 [ 7904.274985] process_one_work+0x3dd/0x9b0 [ 7904.276150] worker_thread+0x296/0x6e0 [ 7904.277117] ? rescuer_thread+0x570/0x570 [ 7904.278529] kthread+0x1d7/0x210 [ 7904.279768] ? set_kthread_struct+0x70/0x70 [ 7904.281385] ret_from_fork+0x1f/0x30 [ 7904.282645] ---[ end trace cbd5b2721f509089 ]--- [ 7904.355594] Lustre: Unmounted lustre-client [ 7928.669524] Key type lgssc unregistered [ 7929.117545] LNet: 139994:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7930.152895] LNet: Removed LNI 192.168.206.7@tcp [ 7931.324253] Key type .llcrypt unregistered [ 7931.327113] Key type ._llcrypt unregistered [ 7950.123496] Key type ._llcrypt registered [ 7950.124770] Key type .llcrypt registered [ 7951.610282] Lustre: DEBUG MARKER: SKIP: conf-sanity test_106 skipping SLOW test 106 [ 7953.524964] Lustre: DEBUG MARKER: == conf-sanity test 107: Unknown config param should not fail target mounting ========================================================== 06:36:47 (1743503807) [ 7956.895882] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7956.914551] alg: No test for adler32 (adler32-zlib) [ 7958.140127] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 7958.442341] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 7958.444844] LNet: Accept secure, port 988 [ 7960.160383] Key type lgssc registered [ 7961.671068] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7994.334339] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7997.618858] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8009.482876] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8033.970209] Key type lgssc unregistered [ 8034.340149] LNet: 143379:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8034.347746] LNet: Removed LNI 192.168.206.7@tcp [ 8035.541190] Key type .llcrypt unregistered [ 8035.542917] Key type ._llcrypt unregistered [ 8046.888587] Key type ._llcrypt registered [ 8046.890209] Key type .llcrypt registered [ 8047.570521] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 8047.588982] alg: No test for adler32 (adler32-zlib) [ 8048.737339] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 8049.023765] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 8049.025451] LNet: Accept secure, port 988 [ 8050.752352] Key type lgssc registered [ 8052.376979] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8130.532783] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8140.526677] Key type lgssc unregistered [ 8140.913945] LNet: 145194:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8140.921424] LNet: Removed LNI 192.168.206.7@tcp [ 8141.903368] Key type .llcrypt unregistered [ 8141.904876] Key type ._llcrypt unregistered [ 8156.933198] Key type ._llcrypt registered [ 8156.934529] Key type .llcrypt registered [ 8158.445912] Lustre: DEBUG MARKER: == conf-sanity test 108a: migrate from ldiskfs to ZFS ==== 06:40:12 (1743504012) [ 8160.094260] Lustre: DEBUG MARKER: SKIP: conf-sanity test_108a zfs only test [ 8161.873337] Lustre: DEBUG MARKER: == conf-sanity test 108b: migrate from ZFS to ldiskfs ==== 06:40:16 (1743504016) [ 8170.726821] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8170.748837] alg: No test for adler32 (adler32-zlib) [ 8171.909244] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 8172.175121] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 8172.177507] LNet: Accept secure, port 988 [ 8173.896314] Key type lgssc registered [ 8175.387453] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8231.390834] Lustre: Mounted lustre-client [ 8231.785777] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 8231.791157] LustreError: 147654:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff926bc94fc000: inode [0x200000bd1:0x2:0x0] mdc close failed: rc = -108 [ 8231.804599] Lustre: 146578:0:(llite_lib.c:4142:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.206.107@tcp:/lustre/fid: [0x240000402:0x3:0x0]/ may get corrupted (rc -108) [ 8231.804820] Lustre: 146579:0:(llite_lib.c:4142:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.206.107@tcp:/lustre/fid: [0x200000bd1:0x2:0x0]/ may get corrupted (rc -108) [ 8242.095199] WARNING: CPU: 2 PID: 28302 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 8242.098769] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 8242.111925] CPU: 2 PID: 28302 Comm: kworker/u8:3 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 8242.114685] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 8242.116983] Workqueue: writeback wb_workfn (flush-lustre-ffff926bc9) [ 8242.118620] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 8242.119998] Code: e8 b8 3b 1b f7 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 8242.125791] RSP: 0018:ffffa5ef827ebb58 EFLAGS: 00010202 [ 8242.127648] RAX: 0000000000000000 RBX: ffffa5ef827ebc40 RCX: 0000000000000001 [ 8242.130173] RDX: 0000000000000007 RSI: ffff926bf92b40c8 RDI: 0000000000000202 [ 8242.132697] RBP: ffff926bedc16310 R08: ffff926bedc16310 R09: 0000000000000001 [ 8242.135244] R10: ffffffffb8177e50 R11: 0000000000000000 R12: ffff926bedc16550 [ 8242.137922] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 8242.140511] FS: 0000000000000000(0000) GS:ffff926c01a00000(0000) knlGS:0000000000000000 [ 8242.143427] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 8242.145495] CR2: 00007f76be9f9298 CR3: 0000000116e28002 CR4: 0000000000170ee0 [ 8242.147100] Call Trace: [ 8242.147672] ? show_regs.cold.9+0x22/0x2f [ 8242.148520] ? __warn+0xc8/0x150 [ 8242.149200] ? ll_writepages+0x3dd/0x400 [lustre] [ 8242.150777] ? report_bug+0x113/0x140 [ 8242.152213] ? ptlrpc_pinger_main+0xaba/0xf20 [ptlrpc] [ 8242.154510] ? do_error_trap+0xb6/0x130 [ 8242.156008] ? do_invalid_op+0x46/0x60 [ 8242.157482] ? ll_writepages+0x3dd/0x400 [lustre] [ 8242.159342] ? invalid_op+0x14/0x20 [ 8242.160725] ? writeback_sb_inodes+0x140/0x7e0 [ 8242.162461] ? ll_writepages+0x3dd/0x400 [lustre] [ 8242.164437] ? ll_writepages+0x3b8/0x400 [lustre] [ 8242.166398] do_writepages+0x7a/0x270 [ 8242.167863] __writeback_single_inode+0xb2/0x7d0 [ 8242.169671] ? do_raw_spin_unlock+0x75/0x190 [ 8242.171333] writeback_sb_inodes+0x239/0x7e0 [ 8242.173115] wb_writeback+0x1ae/0x7a0 [ 8242.174566] wb_workfn+0x1e5/0x950 [ 8242.175934] ? lock_acquire+0x146/0x3a0 [ 8242.177461] ? process_one_work+0x2c0/0x9b0 [ 8242.179111] process_one_work+0x3dd/0x9b0 [ 8242.180734] worker_thread+0x296/0x6e0 [ 8242.182212] ? rescuer_thread+0x570/0x570 [ 8242.183788] kthread+0x1d7/0x210 [ 8242.185060] ? set_kthread_struct+0x70/0x70 [ 8242.186685] ret_from_fork+0x1f/0x30 [ 8242.188108] ---[ end trace cbd5b2721f50908a ]--- [ 8242.262158] Lustre: Unmounted lustre-client [ 8295.457733] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:42:29 (1743504149) [ 8372.371655] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8376.086345] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8390.007087] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8390.748549] Lustre: Mounted lustre-client [ 8459.263257] Lustre: Unmounted lustre-client [ 8509.569486] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8512.728902] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8524.627296] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8525.249734] Lustre: Mounted lustre-client [ 8542.837849] Lustre: Unmounted lustre-client [ 8574.790150] Key type lgssc unregistered [ 8575.221556] LNet: 154210:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8576.297190] LNet: Removed LNI 192.168.206.7@tcp [ 8577.551607] Key type .llcrypt unregistered [ 8577.553261] Key type ._llcrypt unregistered [ 8595.084053] Key type ._llcrypt registered [ 8595.085457] Key type .llcrypt registered [ 8596.456923] Lustre: DEBUG MARKER: == conf-sanity test 109b: test lctl clear_conf one config ========================================================== 06:47:30 (1743504450) [ 8615.637156] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8615.657542] alg: No test for adler32 (adler32-zlib) [ 8616.801702] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 8617.082077] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 8617.083962] LNet: Accept secure, port 988 [ 8618.784287] Key type lgssc registered [ 8620.358520] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8677.345726] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8680.883686] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8694.497466] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8696.136376] Lustre: Mounted lustre-client [ 8763.126261] Lustre: Unmounted lustre-client [ 8831.082868] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8834.692330] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8846.550160] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8847.113819] Lustre: Mounted lustre-client [ 8870.676794] Lustre: Unmounted lustre-client [ 8899.462804] Key type lgssc unregistered [ 8899.955336] LNet: 161250:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8900.967320] LNet: Removed LNI 192.168.206.7@tcp [ 8902.047135] Key type .llcrypt unregistered [ 8902.048063] Key type ._llcrypt unregistered [ 8920.190067] Key type ._llcrypt registered [ 8920.191566] Key type .llcrypt registered [ 8921.758972] Lustre: DEBUG MARKER: == conf-sanity test 110: Adding large_dir with 3-level htree ========================================================== 06:52:56 (1743504776) [ 8932.533769] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8932.571742] alg: No test for adler32 (adler32-zlib) [ 8933.857730] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 8934.127539] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 8934.129231] LNet: Accept secure, port 988 [ 8935.824366] Key type lgssc registered [ 8937.617890] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9003.844291] Lustre: Mounted lustre-client [ 9006.789751] Lustre: Mounted lustre-client [ 9038.304346] WARNING: CPU: 0 PID: 143470 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 9038.309147] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 9038.323779] CPU: 0 PID: 143470 Comm: kworker/u8:0 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 9038.327576] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 9038.330568] Workqueue: writeback wb_workfn (flush-lustre-ffff926bff) [ 9038.332859] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 9038.334781] Code: e8 b8 3b 1b f7 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 9038.341032] RSP: 0018:ffffa5ef8894bb18 EFLAGS: 00010202 [ 9038.342798] RAX: 0000000000000000 RBX: ffffa5ef8894bc00 RCX: 0000000000000001 [ 9038.345150] RDX: 000000000000000e RSI: ffff926bc9f590c8 RDI: 0000000000000202 [ 9038.347485] RBP: ffff926be704c290 R08: 0000000000000000 R09: 0000000000000001 [ 9038.349828] R10: ffffffffb8177e50 R11: 0000000000000000 R12: ffff926be704c4d0 [ 9038.352192] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000 [ 9038.354593] FS: 0000000000000000(0000) GS:ffff926c01200000(0000) knlGS:0000000000000000 [ 9038.357329] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 9038.359454] CR2: 00007fae4349a000 CR3: 0000000116e28004 CR4: 0000000000170ef0 [ 9038.362049] Call Trace: [ 9038.362999] ? show_regs.cold.9+0x22/0x2f [ 9038.364509] ? __warn+0xc8/0x150 [ 9038.365793] ? ll_writepages+0x3dd/0x400 [lustre] [ 9038.367521] ? report_bug+0x113/0x140 [ 9038.369047] ? do_error_trap+0xb6/0x130 [ 9038.370619] ? do_invalid_op+0x46/0x60 [ 9038.371892] ? ll_writepages+0x3dd/0x400 [lustre] [ 9038.373709] ? invalid_op+0x14/0x20 [ 9038.375048] ? writeback_sb_inodes+0x140/0x7e0 [ 9038.376925] ? ll_writepages+0x3dd/0x400 [lustre] [ 9038.378775] ? ll_writepages+0x3b8/0x400 [lustre] [ 9038.380598] do_writepages+0x7a/0x270 [ 9038.381849] __writeback_single_inode+0xb2/0x7d0 [ 9038.383360] ? do_raw_spin_unlock+0x75/0x190 [ 9038.385220] writeback_sb_inodes+0x239/0x7e0 [ 9038.386731] __writeback_inodes_wb+0x71/0x110 [ 9038.388416] wb_writeback+0x627/0x7a0 [ 9038.389767] ? get_nr_inodes+0x45/0x70 [ 9038.391070] wb_workfn+0x778/0x950 [ 9038.392371] ? sched_clock_cpu+0x24/0x150 [ 9038.393741] process_one_work+0x3dd/0x9b0 [ 9038.395111] worker_thread+0x296/0x6e0 [ 9038.396361] ? rescuer_thread+0x570/0x570 [ 9038.397965] kthread+0x1d7/0x210 [ 9038.399389] ? set_kthread_struct+0x70/0x70 [ 9038.400982] ret_from_fork+0x1f/0x30 [ 9038.402316] ---[ end trace cbd5b2721f50908b ]--- [ 9324.079857] Lustre: Unmounted lustre-client [ 9325.019852] Lustre: Unmounted lustre-client [ 9419.238826] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9422.334526] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9435.672323] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9461.876939] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9463.880364] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 07:01:58 (1743505318) [ 9484.268896] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9487.908793] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9499.569972] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9511.168963] Lustre: Mounted lustre-client [ 9516.861968] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff926bef9d9000.ost_server_uuid 50 [ 9518.480028] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff926bef9d9000.ost_server_uuid in FULL state after 0 sec [ 9521.766932] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff926bef9d9000.ost_server_uuid 50 [ 9523.403454] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff926bef9d9000.ost_server_uuid in FULL state after 0 sec [ 9536.999784] Lustre: lustre-OST0001-osc-ffff926bef9d9000: Connection to lustre-OST0001 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 9545.054909] Lustre: Unmounted lustre-client [ 9568.906407] Key type lgssc unregistered [ 9569.287903] LNet: 172080:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9570.343787] LNet: Removed LNI 192.168.206.7@tcp [ 9571.345567] Key type .llcrypt unregistered [ 9571.346476] Key type ._llcrypt unregistered [ 9587.356179] Key type ._llcrypt registered [ 9587.357716] Key type .llcrypt registered [ 9588.845414] Lustre: DEBUG MARKER: == conf-sanity test 112b: mount MDT with no_create option ========================================================== 07:04:03 (1743505443) [ 9591.907164] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9591.927081] alg: No test for adler32 (adler32-zlib) [ 9593.095068] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 9593.379209] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 9593.381582] LNet: Accept secure, port 988 [ 9595.080305] Key type lgssc registered [ 9596.680633] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9621.549906] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9640.291212] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9651.812574] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 9653.467397] Lustre: Mounted lustre-client [ 9746.604924] Lustre: Unmounted lustre-client [ 9771.763898] Key type lgssc unregistered [ 9772.272555] LNet: 175722:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9773.288605] LNet: Removed LNI 192.168.206.7@tcp [ 9774.539144] Key type .llcrypt unregistered [ 9774.540280] Key type ._llcrypt unregistered [ 9794.182564] Key type ._llcrypt registered [ 9794.183646] Key type .llcrypt registered [ 9795.777164] Lustre: DEBUG MARKER: == conf-sanity test 113: Shadow mountpoint correctly report ro/rw for mounts ========================================================== 07:07:30 (1743505650) [ 9814.031426] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9814.061280] alg: No test for adler32 (adler32-zlib) [ 9815.273621] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 9815.592745] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [ 9815.595168] LNet: Accept secure, port 988 [ 9817.336400] Key type lgssc registered [ 9819.063231] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9873.653772] Lustre: Mounted lustre-client [ 9879.570667] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9899.489151] Lustre: lustre-OST0000-osc-ffff926bd4a51000: disconnect after 23s idle [ 9901.576664] Lustre: Unmounted lustre-client [ 9988.342293] Lustre: Mounted lustre-client [ 9993.334855] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10002.334273] Lustre: Unmounted lustre-client [10039.546840] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10041.499796] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10043.328783] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 07:11:37 (1743505897) [10158.156656] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10161.733357] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10175.687488] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10211.560264] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:14:25 (1743506065) [10230.707627] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10234.626954] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10247.545792] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10248.627970] Lustre: Mounted lustre-client [10253.238566] Lustre: Unmounted lustre-client [10278.070869] Key type lgssc unregistered [10278.478835] LNet: 188797:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10279.528312] LNet: Removed LNI 192.168.206.7@tcp [10281.008067] Key type .llcrypt unregistered [10281.009308] Key type ._llcrypt unregistered [10299.272663] Key type ._llcrypt registered [10299.273776] Key type .llcrypt registered [10300.824314] Lustre: DEBUG MARKER: == conf-sanity test 119: writeconf on slave mdt shouldn't duplicate mdc/osp and crash ========================================================== 07:15:55 (1743506155) [10305.774598] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10305.795162] alg: No test for adler32 (adler32-zlib) [10307.001628] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [10307.338065] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [10307.340885] LNet: Accept secure, port 988 [10309.072306] Key type lgssc registered [10310.717985] Lustre: Echo OBD driver; http://www.lustre.org/ [10346.204062] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10350.592950] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10364.080347] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10366.034035] Lustre: Mounted lustre-client [10371.045368] Lustre: lustre-MDT0001-mdc-ffff926c004fc000: Connection to lustre-MDT0001 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [10388.989782] LustreError: 192013:0:(obd_config.c:1995:class_config_llog_handler()) MGC192.168.206.107@tcp: cfg command failed: rc = -17 [10388.993107] Lustre: cmd=cf001 0:lustre-MDT0001-mdc 1:mdc 2:lustre-clilmv_UUID [10388.993107] [10388.997614] LustreError: 191663:0:(mgc_request.c:616:do_requeue()) failed processing log: -17 [10691.557614] Lustre: lustre-MDT0001-mdc-ffff926c004fc000: Connection to lustre-MDT0001 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [10708.988876] LustreError: 193862:0:(obd_config.c:686:class_setup()) Device 4 already setup (type mdc) [10708.996238] LustreError: 193862:0:(obd_config.c:1995:class_config_llog_handler()) MGC192.168.206.107@tcp: cfg command failed: rc = -17 [10709.005601] Lustre: cmd=cf003 0:lustre-MDT0001-mdc 1:lustre-MDT0001_UUID 2:192.168.206.107@tcp [10709.005601] [10709.015584] LustreError: 191663:0:(mgc_request.c:616:do_requeue()) failed processing log: -17 [11008.998525] Lustre: lustre-MDT0001-mdc-ffff926c004fc000: Connection to lustre-MDT0001 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [11027.462442] LustreError: 195690:0:(obd_config.c:1995:class_config_llog_handler()) MGC192.168.206.107@tcp: cfg command failed: rc = -17 [11027.467520] Lustre: cmd=cf014 0:lustre-clilmv 1:lustre-MDT0001_UUID 2:1 3:1 4:lustre-MDT0001-mdc_UUID [11027.467520] [11027.473775] LustreError: 191663:0:(mgc_request.c:616:do_requeue()) failed processing log: -17 [11329.848901] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [11340.208461] Lustre: Unmounted lustre-client [11410.127831] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 07:34:24 (1743507264) [11432.551259] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11436.109111] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11450.644565] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11451.656116] Lustre: Mounted lustre-client [11455.306370] Lustre: Unmounted lustre-client [11491.124963] Lustre: DEBUG MARKER: == conf-sanity test 121: failover MGS ==================== 07:35:45 (1743507345) [11518.461772] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11522.051462] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11549.631655] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mgc.*.mgs_server_uuid [11582.940865] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11586.685696] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11614.256950] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mgc.*.mgs_server_uuid [11634.832751] Lustre: DEBUG MARKER: == conf-sanity test 122a: Check OST sequence update ====== 07:38:09 (1743507489) [11718.985433] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11722.191190] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11735.506518] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11736.429432] Lustre: Mounted lustre-client [11747.077750] Lustre: Unmounted lustre-client [11772.262138] Key type lgssc unregistered [11772.699228] LNet: 210249:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11773.735961] LNet: Removed LNI 192.168.206.7@tcp [11774.852901] Key type .llcrypt unregistered [11774.853920] Key type ._llcrypt unregistered [11794.334701] Key type ._llcrypt registered [11794.336420] Key type .llcrypt registered [11795.771379] Lustre: DEBUG MARKER: == conf-sanity test 123aa: llog_print works with FIDs and simple names ========================================================== 07:40:50 (1743507650) [11799.210821] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11799.234344] alg: No test for adler32 (adler32-zlib) [11800.604606] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [11800.974174] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [11800.976109] LNet: Accept secure, port 988 [11802.792346] Key type lgssc registered [11805.090385] Lustre: Echo OBD driver; http://www.lustre.org/ [11839.294429] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11843.124474] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11855.913541] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11857.745727] Lustre: Mounted lustre-client [11873.545560] Lustre: DEBUG MARKER: == conf-sanity test 123ab: llog_print params output values from set_param -P ========================================================== 07:42:07 (1743507727) [11884.892459] Lustre: DEBUG MARKER: == conf-sanity test 123ac: llog_print with --start and --end ========================================================== 07:42:18 (1743507738) [11894.727474] Lustre: DEBUG MARKER: == conf-sanity test 123ad: llog_print shows all records == 07:42:29 (1743507749) [11905.822925] Lustre: DEBUG MARKER: == conf-sanity test 123ae: llog_cancel can cancel requested record ========================================================== 07:42:39 (1743507759) [11926.573880] Lustre: DEBUG MARKER: == conf-sanity test 123af: llog_catlist can show all config files correctly ========================================================== 07:43:00 (1743507780) [11940.342303] Lustre: DEBUG MARKER: == conf-sanity test 123ag: llog_print skips values deleted by set_param -P -d ========================================================== 07:43:14 (1743507794) [11954.367558] Lustre: DEBUG MARKER: == conf-sanity test 123ah: del_ost cancels config log entries correctly ========================================================== 07:43:28 (1743507808) [11960.138870] Lustre: Unmounted lustre-client [11960.744724] Lustre: Mounted lustre-client [11962.176254] Lustre: Unmounted lustre-client [11994.325314] Key type lgssc unregistered [11994.844371] LNet: 218372:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11995.880143] LNet: Removed LNI 192.168.206.7@tcp [11997.012299] Key type .llcrypt unregistered [11997.013472] Key type ._llcrypt unregistered [12025.375909] Key type ._llcrypt registered [12025.377111] Key type .llcrypt registered [12025.789480] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [12025.829829] alg: No test for adler32 (adler32-zlib) [12027.144459] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [12027.586973] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [12027.588666] LNet: Accept secure, port 988 [12029.392256] Key type lgssc registered [12031.647289] Lustre: Echo OBD driver; http://www.lustre.org/ [12092.737744] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12096.831661] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12110.948803] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12144.458244] Lustre: DEBUG MARKER: == conf-sanity test 123ai: llog_print display all non skipped records ========================================================== 07:46:38 (1743507998) [12162.407167] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12166.061692] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12178.563770] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12180.604521] Lustre: Mounted lustre-client [12264.380905] Lustre: DEBUG MARKER: == conf-sanity test 123F: clear and reset all parameters using set_param -F ========================================================== 07:48:38 (1743508118) [12270.498527] Lustre: Unmounted lustre-client [12313.845970] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12316.847381] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12328.870307] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12329.562648] Lustre: Mounted lustre-client [12332.600540] Lustre: Unmounted lustre-client [12368.559455] Key type lgssc unregistered [12369.093759] LNet: 228096:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12370.154471] LNet: Removed LNI 192.168.206.7@tcp [12371.604144] Key type .llcrypt unregistered [12371.605660] Key type ._llcrypt unregistered [12389.574165] Key type ._llcrypt registered [12389.575215] Key type .llcrypt registered [12391.043120] Lustre: DEBUG MARKER: == conf-sanity test 123G: clear and reset all parameters using apply_yaml ========================================================== 07:50:45 (1743508245) [12396.493328] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12396.515750] alg: No test for adler32 (adler32-zlib) [12397.867822] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [12398.272264] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [12398.274630] LNet: Accept secure, port 988 [12400.096485] Key type lgssc registered [12402.470471] Lustre: Echo OBD driver; http://www.lustre.org/ [12438.410201] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12442.194694] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12454.192603] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12455.826515] Lustre: Mounted lustre-client [12466.707105] Lustre: Unmounted lustre-client [12515.399290] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12519.222789] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12532.615927] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12533.627394] Lustre: Mounted lustre-client [12537.217788] Lustre: Unmounted lustre-client [12563.325904] Key type lgssc unregistered [12564.046400] LNet: 233250:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12565.095880] LNet: Removed LNI 192.168.206.7@tcp [12566.200316] Key type .llcrypt unregistered [12566.201245] Key type ._llcrypt unregistered [12585.632693] Key type ._llcrypt registered [12585.633974] Key type .llcrypt registered [12587.413190] Lustre: DEBUG MARKER: == conf-sanity test 123H: check concurent accesses with 'lctl llog_print ========================================================== 07:54:01 (1743508441) [12590.830422] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12590.850378] alg: No test for adler32 (adler32-zlib) [12592.238232] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [12592.641219] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [12592.643040] LNet: Accept secure, port 988 [12594.368376] Key type lgssc registered [12596.320898] Lustre: Echo OBD driver; http://www.lustre.org/ [12631.611309] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12634.689200] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12646.705151] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12648.480951] Lustre: Mounted lustre-client [12713.512843] Lustre: DEBUG MARKER: == conf-sanity test 124: check failover after replace_nids ========================================================== 07:56:07 (1743508567) [12715.906962] Lustre: DEBUG MARKER: SKIP: conf-sanity test_124 needs MDT failover setup [12718.549630] Lustre: DEBUG MARKER: == conf-sanity test 126: mount in parallel shouldn't cause a crash ========================================================== 07:56:12 (1743508572) [12719.996186] Lustre: Unmounted lustre-client [12743.340923] Key type lgssc unregistered [12743.756881] LNet: 238329:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12744.807938] LNet: Removed LNI 192.168.206.7@tcp [12745.829197] Key type .llcrypt unregistered [12745.830197] Key type ._llcrypt unregistered [12775.442160] Key type ._llcrypt registered [12775.444459] Key type .llcrypt registered [12775.929270] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [12775.963147] alg: No test for adler32 (adler32-zlib) [12777.227756] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [12777.631931] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [12777.633973] LNet: Accept secure, port 988 [12779.416296] Key type lgssc registered [12781.037400] Lustre: Echo OBD driver; http://www.lustre.org/ [12811.173579] Lustre: DEBUG MARKER: == conf-sanity test 127: direct io overwrite on full ost ========================================================== 07:57:45 (1743508665) [12822.779503] Key type lgssc unregistered [12823.344954] LNet: 240602:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12823.355865] LNet: Removed LNI 192.168.206.7@tcp [12824.774481] Key type .llcrypt unregistered [12824.775528] Key type ._llcrypt unregistered [12841.256858] Key type ._llcrypt registered [12841.257888] Key type .llcrypt registered [12841.576204] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [12841.596078] alg: No test for adler32 (adler32-zlib) [12842.777369] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [12843.056667] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [12843.058530] LNet: Accept secure, port 988 [12844.776314] Key type lgssc registered [12847.024871] Lustre: Echo OBD driver; http://www.lustre.org/ [12881.863791] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12884.911182] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12896.586278] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12898.477835] Lustre: Mounted lustre-client [12935.345622] Lustre: DEBUG MARKER: == conf-sanity test 128: Force using remote logs with --nolocallogs ========================================================== 07:59:50 (1743508790) [12936.952907] Lustre: DEBUG MARKER: SKIP: conf-sanity test_128 need separate mgs device [12938.622724] Lustre: DEBUG MARKER: == conf-sanity test 129: attempt to connect an OST with the same index should fail ========================================================== 07:59:53 (1743508793) [12939.232586] WARNING: CPU: 1 PID: 232420 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [12939.234610] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [12939.243469] CPU: 1 PID: 232420 Comm: kworker/u8:0 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [12939.245448] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [12939.247002] Workqueue: writeback wb_workfn (flush-lustre-ffff926bef) [12939.249451] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [12939.250591] Code: e8 b8 3b 1b f7 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [12939.254041] RSP: 0018:ffffa5ef88a33b18 EFLAGS: 00010202 [12939.255060] RAX: 0000000000000000 RBX: ffffa5ef88a33c00 RCX: 0000000000000001 [12939.256332] RDX: 000000000000000e RSI: ffff926becf340c8 RDI: 0000000000000206 [12939.257650] RBP: ffff926bf9796310 R08: 0000000000000000 R09: 0000000000000001 [12939.258989] R10: 000000003f73f0d9 R11: ffff926bee929000 R12: ffff926bf9796550 [12939.260351] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000 [12939.261701] FS: 0000000000000000(0000) GS:ffff926c01600000(0000) knlGS:0000000000000000 [12939.263184] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [12939.264213] CR2: 00005625cd618788 CR3: 0000000116e28005 CR4: 0000000000170ee0 [12939.265568] Call Trace: [12939.266077] ? show_regs.cold.9+0x22/0x2f [12939.266857] ? __warn+0xc8/0x150 [12939.267528] ? ll_writepages+0x3dd/0x400 [lustre] [12939.268564] ? report_bug+0x113/0x140 [12939.269316] ? do_error_trap+0xb6/0x130 [12939.270064] ? do_invalid_op+0x46/0x60 [12939.270793] ? ll_writepages+0x3dd/0x400 [lustre] [12939.271814] ? invalid_op+0x14/0x20 [12939.272597] ? ll_writepages+0x3dd/0x400 [lustre] [12939.273605] ? ll_writepages+0x3b8/0x400 [lustre] [12939.274609] do_writepages+0x7a/0x270 [12939.275270] __writeback_single_inode+0xb2/0x7d0 [12939.276172] ? do_raw_spin_unlock+0x75/0x190 [12939.277011] writeback_sb_inodes+0x239/0x7e0 [12939.277949] __writeback_inodes_wb+0x71/0x110 [12939.278865] wb_writeback+0x627/0x7a0 [12939.279635] ? get_nr_inodes+0x45/0x70 [12939.280443] wb_workfn+0x778/0x950 [12939.281144] ? sched_clock_cpu+0x24/0x150 [12939.282053] process_one_work+0x3dd/0x9b0 [12939.282911] worker_thread+0x296/0x6e0 [12939.283689] ? rescuer_thread+0x570/0x570 [12939.284468] kthread+0x1d7/0x210 [12939.285132] ? set_kthread_struct+0x70/0x70 [12939.285999] ret_from_fork+0x1f/0x30 [12939.286845] ---[ end trace cbd5b2721f50908c ]--- [12939.596370] Lustre: Unmounted lustre-client [12982.976897] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12986.847384] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [13030.811667] Lustre: DEBUG MARKER: == conf-sanity test 130: re-register an MDT after writeconf ========================================================== 08:01:25 (1743508885) [13084.315723] Lustre: Mounted lustre-client [13088.880650] Lustre: DEBUG MARKER: Using TIMEOUT=20 [13100.007336] Lustre: lustre-MDT0001-mdc-ffff926bc9f01000: Connection to lustre-MDT0001 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [13110.241188] Lustre: lustre-OST0000-osc-ffff926bc9f01000: disconnect after 24s idle [13113.402567] LustreError: 248694:0:(obd_config.c:1995:class_config_llog_handler()) MGC192.168.206.107@tcp: cfg command failed: rc = -17 [13113.405124] Lustre: cmd=cf001 0:lustre-MDT0001-mdc 1:mdc 2:lustre-clilmv_UUID [13113.405124] [13113.408166] LustreError: 247986:0:(mgc_request.c:616:do_requeue()) failed processing log: -17 [13125.455172] Lustre: DEBUG MARKER: == conf-sanity test 131: MDT backup restore with project ID ========================================================== 08:02:59 (1743508979) [13173.676540] Lustre: DEBUG MARKER: Using TIMEOUT=20 [13243.118385] Lustre: Unmounted lustre-client [13359.903344] Lustre: Mounted lustre-client [13365.695898] Lustre: DEBUG MARKER: Using TIMEOUT=20 [13403.088618] Lustre: DEBUG MARKER: == conf-sanity test 132: hsm_actions processed after failover ========================================================== 08:07:37 (1743509257) [13403.921185] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [13427.097901] Lustre: Unmounted lustre-client [13532.773453] Lustre: DEBUG MARKER: == conf-sanity test 133: stripe QOS: free space balance in a pool ========================================================== 08:09:46 (1743509386) [13534.878511] Lustre: DEBUG MARKER: SKIP: conf-sanity test_133 needs >= 4 OSTs [13537.433503] Lustre: DEBUG MARKER: == conf-sanity test 134: check_iam works without faults == 08:09:51 (1743509391) [13595.123992] Lustre: Mounted lustre-client [13600.231107] Lustre: DEBUG MARKER: Using TIMEOUT=20 [13931.481212] Lustre: DEBUG MARKER: == conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 08:16:26 (1743509786) [13932.777872] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [13932.780049] Lustre: Skipped 3 previous similar messages [13943.127706] Lustre: Unmounted lustre-client [14048.491798] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [14051.518234] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [14063.736916] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [14064.384342] Lustre: Mounted lustre-client [14180.387584] Lustre: 278879:0:(llog_cat.c:968:llog_cat_process_or_fork()) lustre-MDT0000-mdc-ffff926bff821000: catlog [0xa:0x5:0x0] crosses index zero [14234.312219] Lustre: Unmounted lustre-client [14258.026311] Key type lgssc unregistered [14258.445892] LNet: 280975:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14259.497364] LNet: Removed LNI 192.168.206.7@tcp [14260.601793] Key type .llcrypt unregistered [14260.602865] Key type ._llcrypt unregistered [14272.410241] Key type ._llcrypt registered [14272.411294] Key type .llcrypt registered [14273.835801] Lustre: DEBUG MARKER: == conf-sanity test 136: don't panic with bad obdecho setup ========================================================== 08:22:08 (1743510128) [14286.189621] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [14286.208913] alg: No test for adler32 (adler32-zlib) [14287.344815] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [14287.619611] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [14287.621506] LNet: Accept secure, port 988 [14289.328257] Key type lgssc registered [14290.940526] Lustre: Echo OBD driver; http://www.lustre.org/ [14348.830328] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [14352.251058] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [14365.439294] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [14372.324296] Lustre: Mounted lustre-client [14383.878384] Lustre: Unmounted lustre-client [14480.550697] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [14485.451858] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [14500.552310] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [14536.601769] Lustre: DEBUG MARKER: == conf-sanity test 137: a new MDT should inherit pools, parameters and failnode ========================================================== 08:26:30 (1743510390) [14539.153748] Lustre: DEBUG MARKER: SKIP: conf-sanity test_137 needs >= 3 MDTs [14542.054867] Lustre: DEBUG MARKER: == conf-sanity test 140: remove_updatelog script actions ========================================================== 08:26:35 (1743510395) [14568.075452] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [14572.039806] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [14586.383818] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [14597.115146] Lustre: Mounted lustre-client [14602.219551] Lustre: lustre-MDT0000-mdc-ffff926beea6c000: Connection to lustre-MDT0000 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [14607.335470] Lustre: lustre-MDT0001-mdc-ffff926beea6c000: Connection to lustre-MDT0001 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [14623.584138] Lustre: 281733:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743510463/real 1743510463] req@00000000a5cb6525 x1828202891783936/t0(0) o400->MGC192.168.206.107@tcp@192.168.206.107@tcp:26/25 lens 224/224 e 0 to 1 dl 1743510479 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [14623.591054] LustreError: MGC192.168.206.107@tcp: Connection to MGS (at 192.168.206.107@tcp) was lost; in progress operations using this service will fail [14744.554166] Lustre: Evicted from MGS (at 192.168.206.107@tcp) after server handle changed from 0xb1dd25354c1345ee to 0xb1dd25354c135ca0 [14744.559335] Lustre: MGC192.168.206.107@tcp: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [14751.854737] LustreError: lustre-MDT0000-mdc-ffff926beea6c000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. [14751.904627] Lustre: lustre-MDT0000-mdc-ffff926beea6c000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [14758.581612] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [14759.912946] LustreError: lustre-MDT0001-mdc-ffff926beea6c000: This client was evicted by lustre-MDT0001; in progress operations using this service will fail. [14759.956279] Lustre: lustre-MDT0001-mdc-ffff926beea6c000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [14760.957824] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [14764.692636] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [14766.969906] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [14773.652561] Lustre: Unmounted lustre-client [14866.890518] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [14869.941759] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [14882.951286] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [14925.018538] Lustre: DEBUG MARKER: == conf-sanity test 150: test setting max_cached_mb to a % ========================================================== 08:32:59 (1743510779) [14945.944362] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [14950.091282] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [14962.997819] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [14963.951767] Lustre: Mounted lustre-client [14966.751622] LustreError: 297360:0:(lproc_llite.c:593:ll_max_cached_mb_seq_write()) lustre: can't set max cache more than 3675 MB [14976.310287] Lustre: DEBUG MARKER: == conf-sanity test 151a: damaged local config doesn't prevent mounting ========================================================== 08:33:50 (1743510830) [14977.733435] Lustre: Unmounted lustre-client [15010.372884] Key type lgssc unregistered [15010.923429] LNet: 298366:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [15011.945812] LNet: Removed LNI 192.168.206.7@tcp [15013.350836] Key type .llcrypt unregistered [15013.351968] Key type ._llcrypt unregistered [15031.269856] Key type ._llcrypt registered [15031.270830] Key type .llcrypt registered [15031.599441] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [15031.638443] alg: No test for adler32 (adler32-zlib) [15033.023122] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [15033.486650] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [15033.488783] LNet: Accept secure, port 988 [15035.320348] Key type lgssc registered [15037.810097] Lustre: Echo OBD driver; http://www.lustre.org/ [15114.048281] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [15118.546655] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [15133.823792] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [15233.214457] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [15237.084959] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [15252.502871] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [15291.252586] Lustre: DEBUG MARKER: == conf-sanity test 151b: -ENOSPC doesn't affect mount === 08:39:05 (1743511145) [15303.009220] Key type lgssc unregistered [15303.473551] LNet: 305059:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [15303.489198] LNet: Removed LNI 192.168.206.7@tcp [15304.717549] Key type .llcrypt unregistered [15304.718711] Key type ._llcrypt unregistered [15321.503509] Key type ._llcrypt registered [15321.504571] Key type .llcrypt registered [15322.003147] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [15322.028178] alg: No test for adler32 (adler32-zlib) [15323.398370] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [15323.813563] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [15323.815715] LNet: Accept secure, port 988 [15325.624584] Key type lgssc registered [15327.452475] Lustre: Echo OBD driver; http://www.lustre.org/ [15368.398392] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [15372.598831] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [15386.391910] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [15474.782665] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [15479.053599] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [15494.102461] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [15528.984410] Lustre: DEBUG MARKER: == conf-sanity test 152: seq allocation error in OSP ===== 08:43:03 (1743511383) [15588.614902] Lustre: Mounted lustre-client [15594.648496] Lustre: DEBUG MARKER: Using TIMEOUT=20 [15612.613831] Lustre: DEBUG MARKER: ADD OST3 [15614.434054] Lustre: lustre-OST0000-osc-ffff926bed349000: disconnect after 23s idle [15623.989241] Lustre: DEBUG MARKER: STOP OST3 [15633.387562] Lustre: lustre-OST0003-osc-ffff926bed349000: Connection to lustre-OST0003 (at 192.168.206.107@tcp) was lost; in progress operations using this service will wait for recovery to complete [15639.055872] Lustre: DEBUG MARKER: START OST3 again [15645.039338] Lustre: lustre-OST0003-osc-ffff926bed349000: Connection restored to 192.168.206.107@tcp (at 192.168.206.107@tcp) [15672.309873] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [15672.315666] LustreError: 315113:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff926bed349000: inode [0x240000bd0:0x1:0x0] mdc close failed: rc = -108 [15682.693804] Lustre: Unmounted lustre-client [15782.512212] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [15787.052838] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [15802.842268] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [15840.066763] Lustre: DEBUG MARKER: == conf-sanity test 153a: bypass invalid NIDs quickly ==== 08:48:14 (1743511694) [15912.033857] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [15915.037326] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [15928.232969] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [15970.316621] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [15973.373278] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [15984.827770] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [15986.228098] LNetError: 305456:0:(lib-move.c:2282:lnet_handle_find_routed_path()) peer 192.168.252.112@tcp2 has no available nets [15986.426319] Lustre: Mounted lustre-client [15992.060953] Lustre: Unmounted lustre-client [16011.961803] Key type lgssc unregistered [16047.201901] LNet: 323515:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [16114.795829] LNet: Removed LNI 192.168.206.7@tcp [16115.933348] Key type .llcrypt unregistered [16115.934306] Key type ._llcrypt unregistered [16136.169202] Key type ._llcrypt registered [16136.170148] Key type .llcrypt registered [16137.701915] Lustre: DEBUG MARKER: == conf-sanity test 153b: added IPv6 NID support ========= 08:53:12 (1743511992) [16153.690442] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [16153.713568] alg: No test for adler32 (adler32-zlib) [16155.060781] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [16155.471793] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [16155.473664] LNet: Accept secure, port 988 [16157.256315] Key type lgssc registered [16159.481726] Lustre: Echo OBD driver; http://www.lustre.org/ [16223.113492] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [16227.129845] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [16240.652498] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [16323.189674] Lustre: Mounted lustre-client [16328.924481] Lustre: DEBUG MARKER: Using TIMEOUT=20 [16345.944415] Lustre: Unmounted lustre-client [16346.129820] LNetError: 324698:0:(lib-socket.c:302:lnet_sock_create()) Error trying to bind to fe80::54b0:97ff:fe32:54d0/1023: rc = -22 [16346.132369] LNetError: Unexpected error -22 connecting to 6699:7654::1234:1234:d84@tcp at host [6699:7654::1234:1234:d84]:988 [16346.562475] Lustre: Mounted lustre-client [16346.946870] Lustre: Unmounted lustre-client [16347.995357] Lustre: Unmounted lustre-client [16348.203493] LNetError: 324699:0:(lib-socket.c:302:lnet_sock_create()) Error trying to bind to fe80::54b0:97ff:fe32:54d0/1023: rc = -22 [16348.206185] LNetError: Unexpected error -22 connecting to 6699:7654::1234:1234:d84@tcp at host [6699:7654::1234:1234:d84]:988 [16348.559065] Lustre: Mounted lustre-client [16348.560052] Lustre: Skipped 1 previous similar message [16349.330491] LNetError: Refusing connection from 192.168.206.7 for 127.0.0.5@tcp: No matching NI [16349.332684] LNetError: 324700:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.5 [16349.335118] LNetError: Connection to 127.0.0.5@tcp at host 127.0.0.5:988 was reset: is it running a compatible version of Lustre and is 127.0.0.5@tcp one of its NIDs? [16350.003975] Lustre: Unmounted lustre-client [16350.005111] Lustre: Skipped 1 previous similar message [16350.189130] LNetError: Refusing connection from 192.168.206.7 for 127.0.0.5@tcp: No matching NI [16350.193162] LNetError: 324697:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.5 [16350.195576] LNetError: Connection to 127.0.0.5@tcp at host 127.0.0.5:988 was reset: is it running a compatible version of Lustre and is 127.0.0.5@tcp one of its NIDs? [16351.308475] Lustre: Mounted lustre-client [16351.309441] Lustre: Skipped 2 previous similar messages [16351.970135] LNetError: 324699:0:(lib-socket.c:302:lnet_sock_create()) Error trying to bind to fe80::54b0:97ff:fe32:54d0/1023: rc = -22 [16351.972762] LNetError: Unexpected error -22 connecting to 6699:7654::1234:1234:d84@tcp at host [6699:7654::1234:1234:d84]:988 [16353.036483] LNetError: Refusing connection from 192.168.206.7 for 127.0.0.5@tcp: No matching NI [16353.038627] LNetError: 324700:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.5 [16353.042124] LNetError: Connection to 127.0.0.5@tcp at host 127.0.0.5:988 was reset: is it running a compatible version of Lustre and is 127.0.0.5@tcp one of its NIDs? [16354.883397] Lustre: Unmounted lustre-client [16354.884438] Lustre: Skipped 4 previous similar messages [16355.077540] LNetError: 324700:0:(lib-socket.c:302:lnet_sock_create()) Error trying to bind to fe80::54b0:97ff:fe32:54d0/1023: rc = -22 [16355.078926] LNetError: Unexpected error -22 connecting to 2001:db8:85a3::8a2e:370:7334@tcp at host [2001:db8:85a3::8a2e:370:7334]:988 [16355.079971] LNetError: 324700:0:(lib-socket.c:302:lnet_sock_create()) Skipped 3 previous similar messages [16355.083328] LNetError: Skipped 2 previous similar messages [16355.424598] Lustre: Mounted lustre-client [16355.425526] Lustre: Skipped 3 previous similar messages [16366.600647] Lustre: DEBUG MARKER: == conf-sanity test 153c: don't stuck on unreached NID === 08:57:00 (1743512220) [16465.215973] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [16468.256981] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [16480.612116] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [16529.399410] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [16533.527285] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [16545.865868] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [16552.929107] LustreError: 335490:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.252.112@tcp: failed processing log, type 1: rc = -5 [16691.168598] LustreError: MGC192.168.252.112@tcp: Confguration from log lustre-client failed from MGS -5. Communication error between node & MGS, a bad configuration, or other errors. See syslog for more info [16691.172984] Lustre: Unmounted lustre-client [16691.173945] Lustre: Skipped 1 previous similar message [16691.183862] LustreError: 335490:0:(super25.c:181:lustre_fill_super()) llite: Unable to mount : rc = -5 [16715.218918] Key type lgssc unregistered [16734.176921] LNet: 335957:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [16807.912975] LNet: Removed LNI 192.168.206.7@tcp [16809.839528] Key type .llcrypt unregistered [16809.840578] Key type ._llcrypt unregistered [16833.281500] Key type ._llcrypt registered [16833.282487] Key type .llcrypt registered [16835.641838] Lustre: DEBUG MARKER: == conf-sanity test 154: expand .. on rename after MDT backup restore ========================================================== 09:04:49 (1743512689) [16854.795342] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [16854.844820] alg: No test for adler32 (adler32-zlib) [16856.335610] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [16856.839406] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [16856.841798] LNet: Accept secure, port 988 [16858.696361] Key type lgssc registered [16861.600540] Lustre: Echo OBD driver; http://www.lustre.org/ [16980.429096] Lustre: Mounted lustre-client [16985.498804] Lustre: DEBUG MARKER: Using TIMEOUT=20 [17000.715668] Lustre: Unmounted lustre-client [17129.487994] Lustre: Mounted lustre-client [17135.358156] Lustre: DEBUG MARKER: Using TIMEOUT=20 [17145.292436] Lustre: Unmounted lustre-client [17191.271965] Lustre: DEBUG MARKER: == conf-sanity test 155: gap in seq allocation from ofd after restarting ========================================================== 09:10:45 (1743513045) [17247.353825] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [17250.174168] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [17265.439877] Lustre: DEBUG MARKER: oleg607-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [17342.525679] Lustre: Mounted lustre-client [17347.203703] Lustre: DEBUG MARKER: Using TIMEOUT=20 [17364.381970] WARNING: CPU: 2 PID: 289033 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [17364.385201] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel ghash_clmulni_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [17364.393917] CPU: 2 PID: 289033 Comm: kworker/u8:0 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [17364.396090] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [17364.397760] Workqueue: writeback wb_workfn (flush-lustre-ffff926bec) [17364.399999] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [17364.401100] Code: e8 b8 3b 1b f7 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [17364.404782] RSP: 0018:ffffa5ef88743b58 EFLAGS: 00010202 [17364.405787] RAX: 0000000000000000 RBX: ffffa5ef88743c40 RCX: 0000000000000001 [17364.407173] RDX: 0000000000000007 RSI: ffff926bd3d310c8 RDI: 0000000000000206 [17364.408541] RBP: ffff926be7013250 R08: 0000000000000000 R09: 0000000000000001 [17364.409929] R10: 00000000707e2086 R11: ffff926bf9100000 R12: ffff926be7013490 [17364.411336] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [17364.412786] FS: 0000000000000000(0000) GS:ffff926c01a00000(0000) knlGS:0000000000000000 [17364.414371] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [17364.415508] CR2: 00005622e5834f44 CR3: 0000000116e28006 CR4: 0000000000170ee0 [17364.416876] Call Trace: [17364.417370] ? show_regs.cold.9+0x22/0x2f [17364.418136] ? __warn+0xc8/0x150 [17364.418756] ? ll_writepages+0x3dd/0x400 [lustre] [17364.419725] ? report_bug+0x113/0x140 [17364.420412] ? do_error_trap+0xb6/0x130 [17364.421173] ? do_invalid_op+0x46/0x60 [17364.421862] ? ll_writepages+0x3dd/0x400 [lustre] [17364.422829] ? invalid_op+0x14/0x20 [17364.423508] ? ll_writepages+0x3dd/0x400 [lustre] [17364.424461] ? ll_writepages+0x3b8/0x400 [lustre] [17364.425455] do_writepages+0x7a/0x270 [17364.426164] __writeback_single_inode+0xb2/0x7d0 [17364.427066] ? do_raw_spin_unlock+0x75/0x190 [17364.427843] writeback_sb_inodes+0x239/0x7e0 [17364.428677] wb_writeback+0x1ae/0x7a0 [17364.429417] wb_workfn+0x1e5/0x950 [17364.430118] ? lock_acquire+0x146/0x3a0 [17364.430881] ? process_one_work+0x2c0/0x9b0 [17364.431703] process_one_work+0x3dd/0x9b0 [17364.432475] worker_thread+0x296/0x6e0 [17364.433231] ? rescuer_thread+0x570/0x570 [17364.434030] kthread+0x1d7/0x210 [17364.434683] ? set_kthread_struct+0x70/0x70 [17364.435514] ret_from_fork+0x1f/0x30 [17364.436183] ---[ end trace cbd5b2721f50908d ]--- [17364.517625] Lustre: Unmounted lustre-client [17463.010144] Lustre: Mounted lustre-client [17468.699932] Lustre: DEBUG MARKER: Using TIMEOUT=20 [17488.867204] Lustre: lustre-OST0001-osc-ffff926c00511000: disconnect after 23s idle [17495.593401] Lustre: DEBUG MARKER: == conf-sanity test 200a: check CPU partitions =========== 09:15:49 (1743513349) [17497.000843] Lustre: Unmounted lustre-client [17536.927358] Key type lgssc unregistered [17537.505463] LNet: 353690:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17538.539295] LNet: Removed LNI 192.168.206.7@tcp [17540.192440] Key type .llcrypt unregistered [17540.193418] Key type ._llcrypt unregistered [17543.995462] Key type ._llcrypt registered [17543.996452] Key type .llcrypt registered [17544.392090] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 4 [17544.437970] alg: No test for adler32 (adler32-zlib) [17545.822256] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17546.239233] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17546.241655] LNet: Accept secure, port 988 [17548.104450] Key type lgssc registered [17550.710605] Lustre: Echo OBD driver; http://www.lustre.org/ [17576.265192] Key type lgssc unregistered [17576.655597] LNet: 355212:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17576.670870] LNet: Removed LNI 192.168.206.7@tcp [17577.739681] Key type .llcrypt unregistered [17577.741100] Key type ._llcrypt unregistered [17578.275598] Key type ._llcrypt registered [17578.277161] Key type .llcrypt registered [17580.382652] Lustre: DEBUG MARKER: == conf-sanity test 200b: set CPU pattern using core selection ========================================================== 09:17:14 (1743513434) [17591.356490] Key type .llcrypt unregistered [17591.357972] Key type ._llcrypt unregistered [17593.741472] Key type ._llcrypt registered [17593.742436] Key type .llcrypt registered [17594.058715] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [17594.084614] alg: No test for adler32 (adler32-zlib) [17595.273716] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17595.574211] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17595.576498] LNet: Accept secure, port 988 [17597.360449] Key type lgssc registered [17599.212444] Lustre: Echo OBD driver; http://www.lustre.org/ [17617.286435] Key type lgssc unregistered [17617.863148] LNet: 356644:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17617.875740] LNet: Removed LNI 192.168.206.7@tcp [17619.557931] Key type .llcrypt unregistered [17619.559067] Key type ._llcrypt unregistered [17637.479507] Key type ._llcrypt registered [17637.480920] Key type .llcrypt registered [17637.940882] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [17637.982812] alg: No test for adler32 (adler32-zlib) [17639.375392] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17639.789605] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17639.792490] LNet: Accept secure, port 988 [17641.656298] Key type lgssc registered [17644.084333] Lustre: Echo OBD driver; http://www.lustre.org/ [17671.280528] Key type lgssc unregistered [17671.778441] LNet: 358292:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17671.790323] LNet: Removed LNI 192.168.206.7@tcp [17673.274423] Key type .llcrypt unregistered [17673.276368] Key type ._llcrypt unregistered [17673.866990] Key type ._llcrypt registered [17673.868134] Key type .llcrypt registered [17675.813279] Lustre: DEBUG MARKER: == conf-sanity test 200c: set CPU pattern using NUMA node layout ========================================================== 09:18:49 (1743513529) [17683.673162] Key type .llcrypt unregistered [17683.674213] Key type ._llcrypt unregistered [17688.464225] Key type ._llcrypt registered [17688.465331] Key type .llcrypt registered [17688.791394] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [17688.833140] alg: No test for adler32 (adler32-zlib) [17690.212373] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17690.679875] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17690.681640] LNet: Accept secure, port 988 [17692.464231] Key type lgssc registered [17694.910571] Lustre: Echo OBD driver; http://www.lustre.org/ [17712.349568] Key type lgssc unregistered [17712.944993] LNet: 359719:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17712.953636] LNet: Removed LNI 192.168.206.7@tcp [17714.360370] Key type .llcrypt unregistered [17714.361369] Key type ._llcrypt unregistered [17726.003588] Key type ._llcrypt registered [17726.004547] Key type .llcrypt registered [17726.189742] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [17726.207418] alg: No test for adler32 (adler32-zlib) [17727.306155] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17727.570931] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17727.572694] LNet: Accept secure, port 988 [17729.288315] Key type lgssc registered [17731.576958] Lustre: Echo OBD driver; http://www.lustre.org/ [17748.742941] Key type lgssc unregistered [17749.284546] LNet: 360909:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17749.292770] LNet: Removed LNI 192.168.206.7@tcp [17750.716499] Key type .llcrypt unregistered [17750.717526] Key type ._llcrypt unregistered [17762.553651] Key type ._llcrypt registered [17762.554624] Key type .llcrypt registered [17763.117943] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [17763.158218] alg: No test for adler32 (adler32-zlib) [17764.533911] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17764.972129] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17764.973829] LNet: Accept secure, port 988 [17766.728341] Key type lgssc registered [17768.882169] Lustre: Echo OBD driver; http://www.lustre.org/ [17796.000662] Key type lgssc unregistered [17796.644419] LNet: 362557:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17796.657890] LNet: Removed LNI 192.168.206.7@tcp [17798.365889] Key type .llcrypt unregistered [17798.366900] Key type ._llcrypt unregistered [17798.921594] Key type ._llcrypt registered [17798.922614] Key type .llcrypt registered [17800.661781] Lustre: DEBUG MARKER: == conf-sanity test 200d: set CPU pattern to exclude only CPU 1 ========================================================== 09:20:54 (1743513654) [17809.330715] Key type .llcrypt unregistered [17809.331967] Key type ._llcrypt unregistered [17813.463737] Key type ._llcrypt registered [17813.464788] Key type .llcrypt registered [17813.913114] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [17813.944753] alg: No test for adler32 (adler32-zlib) [17815.328203] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17815.777340] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17815.779598] LNet: Accept secure, port 988 [17817.624357] Key type lgssc registered [17820.066741] Lustre: Echo OBD driver; http://www.lustre.org/ [17836.596842] Key type lgssc unregistered [17837.104838] LNet: 363989:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17837.113419] LNet: Removed LNI 192.168.206.7@tcp [17838.364027] Key type .llcrypt unregistered [17838.365052] Key type ._llcrypt unregistered [17849.485894] Key type ._llcrypt registered [17849.486754] Key type .llcrypt registered [17849.760147] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [17849.778206] alg: No test for adler32 (adler32-zlib) [17850.882219] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17851.132599] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17851.134092] LNet: Accept secure, port 988 [17852.824228] Key type lgssc registered [17854.386181] Lustre: Echo OBD driver; http://www.lustre.org/ [17867.465836] Key type lgssc unregistered [17867.861845] LNet: 365183:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17867.873797] LNet: Removed LNI 192.168.206.7@tcp [17868.866898] Key type .llcrypt unregistered [17868.867953] Key type ._llcrypt unregistered [17876.847875] Key type ._llcrypt registered [17876.850090] Key type .llcrypt registered [17877.347268] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [17877.368857] alg: No test for adler32 (adler32-zlib) [17878.531913] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17878.819606] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17878.821562] LNet: Accept secure, port 988 [17880.536234] Key type lgssc registered [17882.193275] Lustre: Echo OBD driver; http://www.lustre.org/ [17895.296706] Key type lgssc unregistered [17895.697971] LNet: 366370:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17895.707861] LNet: Removed LNI 192.168.206.7@tcp [17896.707762] Key type .llcrypt unregistered [17896.708710] Key type ._llcrypt unregistered [17906.450213] Key type ._llcrypt registered [17906.451176] Key type .llcrypt registered [17906.941992] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [17906.977941] alg: No test for adler32 (adler32-zlib) [17908.282731] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17908.645847] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17908.647506] LNet: Accept secure, port 988 [17910.408348] Key type lgssc registered [17912.046411] Lustre: Echo OBD driver; http://www.lustre.org/ [17935.630402] Key type lgssc unregistered [17936.058814] LNet: 368023:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17936.072344] LNet: Removed LNI 192.168.206.7@tcp [17937.210330] Key type .llcrypt unregistered [17937.211328] Key type ._llcrypt unregistered [17937.659258] Key type ._llcrypt registered [17937.660570] Key type .llcrypt registered [17939.422689] Lustre: DEBUG MARKER: == conf-sanity test 200e: set CPU pattern using relative core exclusion ========================================================== 09:23:13 (1743513793) [17948.969302] Key type .llcrypt unregistered [17948.970590] Key type ._llcrypt unregistered [17952.760879] Key type ._llcrypt registered [17952.761957] Key type .llcrypt registered [17953.298909] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [17953.322585] alg: No test for adler32 (adler32-zlib) [17954.627847] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17955.051806] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17955.053821] LNet: Accept secure, port 988 [17956.872360] Key type lgssc registered [17959.215335] Lustre: Echo OBD driver; http://www.lustre.org/ [17977.663536] Key type lgssc unregistered [17978.226541] LNet: 369451:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [17978.238628] LNet: Removed LNI 192.168.206.7@tcp [17979.784152] Key type .llcrypt unregistered [17979.785258] Key type ._llcrypt unregistered [17993.168448] Key type ._llcrypt registered [17993.170126] Key type .llcrypt registered [17993.779981] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [17993.808814] alg: No test for adler32 (adler32-zlib) [17995.359874] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [17995.884609] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [17995.886642] LNet: Accept secure, port 988 [17997.760465] Key type lgssc registered [18000.754497] Lustre: Echo OBD driver; http://www.lustre.org/ [18019.469194] Key type lgssc unregistered [18019.983971] LNet: 370652:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [18019.994574] LNet: Removed LNI 192.168.206.7@tcp [18021.413776] Key type .llcrypt unregistered [18021.414812] Key type ._llcrypt unregistered [18029.851941] Key type ._llcrypt registered [18029.913111] Key type .llcrypt registered [18030.288778] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [18030.312738] alg: No test for adler32 (adler32-zlib) [18031.685651] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [18032.095590] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [18032.097427] LNet: Accept secure, port 988 [18033.840290] Key type lgssc registered [18035.412423] Lustre: Echo OBD driver; http://www.lustre.org/ [18049.905903] Key type lgssc unregistered [18050.275843] LNet: 371842:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [18050.285198] LNet: Removed LNI 192.168.206.7@tcp [18051.283332] Key type .llcrypt unregistered [18051.284296] Key type ._llcrypt unregistered [18063.871107] Key type ._llcrypt registered [18063.872220] Key type .llcrypt registered [18064.300237] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [18064.333135] alg: No test for adler32 (adler32-zlib) [18065.780089] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [18066.219854] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [18066.221691] LNet: Accept secure, port 988 [18068.048238] Key type lgssc registered [18070.574727] Lustre: Echo OBD driver; http://www.lustre.org/ [18100.392088] Key type lgssc unregistered [18100.877438] LNet: 373522:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [18100.891380] LNet: Removed LNI 192.168.206.7@tcp [18102.216647] Key type .llcrypt unregistered [18102.217703] Key type ._llcrypt unregistered [18102.709399] Key type ._llcrypt registered [18102.710498] Key type .llcrypt registered [18104.703872] Lustre: DEBUG MARKER: == conf-sanity test 802a: simulate readonly device ======= 09:25:58 (1743513958) [18107.090185] Lustre: DEBUG MARKER: SKIP: conf-sanity test_802a ZFS specific test [18124.860621] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [18124.885817] alg: No test for adler32 (adler32-zlib) [18126.257740] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [18126.682269] LNet: Added LNI 192.168.206.7@tcp [8/256/0/180] [18126.684699] LNet: Accept secure, port 988 [18128.480350] Key type lgssc registered [18130.964602] Lustre: Echo OBD driver; http://www.lustre.org/ [18177.146155] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 17589 sec ========= 09:27:11 (1743514031) [18179.658960] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 09:27:13 (1743514033) === [18185.114228] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 09:27:18 (1743514038) === [18208.443172] Key type lgssc unregistered [18208.990473] LNet: 377133:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [18208.998457] LNet: Removed LNI 192.168.206.7@tcp [18210.395506] Key type .llcrypt unregistered [18210.396464] Key type ._llcrypt unregistered