[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-22) (GCC)) #7 SMP Sat Jan 18 21:01:29 EST 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0.0 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 489734970 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5410-0x000f541f] [ 0.000000] RAMDISK: [mem 0xbcbf3000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5220 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1D6F 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1C0B 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001BCB (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1C7F 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1D0F 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1D47 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1c0b-0xbffe1c7e] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1c0a] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1c7f-0xbffe1d0e] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1d0f-0xbffe1d46] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1d47-0xbffe1d6e] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 513 pages/cpu s2064384 r8192 d28672 u4194304 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2819008K/4306400K available (20483K kernel code, 12066K rwdata, 7356K rodata, 4680K init, 23504K bss, 542472K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41388 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x616/0x99a with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar [ 0.001000] ... MAX_LOCKDEP_SUBCLASSES: 8 [ 0.001000] ... MAX_LOCK_DEPTH: 48 [ 0.001000] ... MAX_LOCKDEP_KEYS: 8192 [ 0.001000] ... CLASSHASH_SIZE: 4096 [ 0.001000] ... MAX_LOCKDEP_ENTRIES: 32768 [ 0.001000] ... MAX_LOCKDEP_CHAINS: 65536 [ 0.001000] ... CHAINHASH_SIZE: 32768 [ 0.001000] memory used by lock dependency info: 4149 kB [ 0.001000] per task-struct memory footprint: 2688 bytes [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001018] APIC: Switch to symmetric I/O mode setup [ 0.002499] x2apic enabled [ 0.003012] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008765] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009033] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010017] pid_max: default: 32768 minimum: 301 [ 0.011381] LSM: Security Framework initializing [ 0.012152] Yama: becoming mindful. [ 0.014012] SELinux: Initializing. [ 0.015285] *** VALIDATE selinux *** [ 0.023213] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028323] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.030248] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031170] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.033304] *** VALIDATE tmpfs *** [ 0.036752] *** VALIDATE proc *** [ 0.038107] *** VALIDATE cgroup *** [ 0.039022] *** VALIDATE cgroup2 *** [ 0.040412] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.041250] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.042013] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.043038] Spectre V2 : User space: Vulnerable [ 0.044011] Speculative Store Bypass: Vulnerable [ 0.046796] debug: unmapping init [mem 0xffffffffa3903000-0xffffffffa390afff] [ 0.048471] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.050931] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.051039] ... version: 2 [ 0.052013] ... bit width: 48 [ 0.053015] ... generic registers: 4 [ 0.054014] ... value mask: 0000ffffffffffff [ 0.055018] ... max period: 00007fffffffffff [ 0.056020] ... fixed-purpose events: 3 [ 0.057018] ... event mask: 000000070000000f [ 0.058574] rcu: Hierarchical SRCU implementation. [ 0.063384] smp: Bringing up secondary CPUs ... [ 0.065741] x86: Booting SMP configuration: [ 0.066024] .... node #0, CPUs: #1 [ 0.071599] #2 [ 0.075766] #3 [ 0.077705] smp: Brought up 1 node, 4 CPUs [ 0.078048] smpboot: Max logical packages: 1 [ 0.079022] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.192675] node 0 deferred pages initialised in 109ms [ 0.193260] pgdatinit0 (35) used greatest stack depth: 14528 bytes left [ 0.198313] devtmpfs: initialized [ 0.200155] x86/mm: Memory block size: 128MB [ 0.213897] gcov: version magic: 0x41383552 [ 0.216979] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.220193] futex hash table entries: 1024 (order: 5, 131072 bytes, vmalloc) [ 0.223948] pinctrl core: initialized pinctrl subsystem [ 0.226988] [ 0.227014] ************************************************************* [ 0.229023] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.232024] ** ** [ 0.233000] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.233020] ** ** [ 0.235023] ** This means that this kernel is built to expose internal ** [ 0.237023] ** IOMMU data structures, which may compromise security on ** [ 0.239023] ** your system. ** [ 0.241018] ** ** [ 0.242015] ** If you see this message and you are not debugging the ** [ 0.244018] ** kernel, report this immediately to your vendor! ** [ 0.246021] ** ** [ 0.248022] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.249017] ************************************************************* [ 0.252469] NET: Registered protocol family 16 [ 0.256384] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.259146] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.261132] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.266226] cpuidle: using governor menu [ 0.268018] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.270349] PCI: Using configuration type 1 for base access [ 0.271140] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.321978] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.322022] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.333461] cryptd: max_cpu_qlen set to 1000 [ 0.338195] ACPI: Added _OSI(Module Device) [ 0.340038] ACPI: Added _OSI(Processor Device) [ 0.341019] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.342018] ACPI: Added _OSI(Processor Aggregator Device) [ 0.381553] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.394081] ACPI: Interpreter enabled [ 0.395284] ACPI: PM: (supports S0 S3 S4 S5) [ 0.397037] ACPI: Using IOAPIC for interrupt routing [ 0.399417] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.405000] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.483000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.486108] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.489027] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.491448] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.504412] acpiphp: Slot [2] registered [ 0.505443] acpiphp: Slot [5] registered [ 0.506313] acpiphp: Slot [6] registered [ 0.507367] acpiphp: Slot [3] registered [ 0.508405] acpiphp: Slot [4] registered [ 0.509335] acpiphp: Slot [7] registered [ 0.510388] acpiphp: Slot [8] registered [ 0.511370] acpiphp: Slot [9] registered [ 0.512385] acpiphp: Slot [10] registered [ 0.513332] acpiphp: Slot [11] registered [ 0.514393] acpiphp: Slot [12] registered [ 0.515384] acpiphp: Slot [13] registered [ 0.516361] acpiphp: Slot [14] registered [ 0.517281] acpiphp: Slot [15] registered [ 0.518296] acpiphp: Slot [16] registered [ 0.519337] acpiphp: Slot [17] registered [ 0.520385] acpiphp: Slot [18] registered [ 0.521315] acpiphp: Slot [19] registered [ 0.522285] acpiphp: Slot [20] registered [ 0.523306] acpiphp: Slot [21] registered [ 0.524344] acpiphp: Slot [22] registered [ 0.525302] acpiphp: Slot [23] registered [ 0.526358] acpiphp: Slot [24] registered [ 0.527363] acpiphp: Slot [25] registered [ 0.528295] acpiphp: Slot [26] registered [ 0.529347] acpiphp: Slot [27] registered [ 0.530379] acpiphp: Slot [28] registered [ 0.531340] acpiphp: Slot [29] registered [ 0.532343] acpiphp: Slot [30] registered [ 0.533386] acpiphp: Slot [31] registered [ 0.534196] PCI host bridge to bus 0000:00 [ 0.535043] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.536051] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.537057] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.538075] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.539048] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x38007fffffff window] [ 0.540052] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.541440] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.544663] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.547747] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.551864] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.554057] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.555034] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.556030] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.557029] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.559833] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.560973] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.561055] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.564194] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.566026] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.571023] pci 0000:00:02.0: reg 0x20: [mem 0x380000000000-0x380000003fff 64bit pref] [ 0.573987] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.594171] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.597030] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.600029] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.607024] pci 0000:00:05.0: reg 0x20: [mem 0x380000004000-0x380000007fff 64bit pref] [ 0.628661] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.631023] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.634023] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.641026] pci 0000:00:06.0: reg 0x20: [mem 0x380000008000-0x38000000bfff 64bit pref] [ 0.668897] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.672051] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.676086] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.680019] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.682947] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.693063] iommu: Default domain type: Passthrough [ 0.696000] SCSI subsystem initialized [ 0.698736] ACPI: bus type USB registered [ 0.700576] usbcore: registered new interface driver usbfs [ 0.703417] usbcore: registered new interface driver hub [ 0.705402] usbcore: registered new device driver usb [ 0.707835] pps_core: LinuxPPS API ver. 1 registered [ 0.710033] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.713202] PTP clock support registered [ 0.718365] EDAC MC: Ver: 3.0.0 [ 0.720643] PCI: Using ACPI for IRQ routing [ 0.725718] NetLabel: Initializing [ 0.726025] NetLabel: domain hash size = 128 [ 0.727020] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.728354] NetLabel: unlabeled traffic allowed by default [ 0.729746] vgaarb: loaded [ 0.731176] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.732020] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.737428] clocksource: Switched to clocksource kvm-clock [ 1.196675] VFS: Disk quotas dquot_6.6.0 [ 1.198317] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.200805] *** VALIDATE ramfs *** [ 1.202194] *** VALIDATE hugetlbfs *** [ 1.204454] pnp: PnP ACPI init [ 1.214190] pnp: PnP ACPI: found 6 devices [ 1.273807] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.277375] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.279355] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.281619] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.283814] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.285796] pci_bus 0000:00: resource 8 [mem 0x380000000000-0x38007fffffff window] [ 1.289331] NET: Registered protocol family 2 [ 1.293080] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.300628] tcp_listen_portaddr_hash hash table entries: 4096 (order: 6, 360448 bytes, vmalloc) [ 1.303486] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.308931] TCP bind hash table entries: 65536 (order: 10, 5242880 bytes, vmalloc) [ 1.318714] TCP: Hash tables configured (established 65536 bind 65536) [ 1.322518] MPTCP token hash table entries: 8192 (order: 7, 786432 bytes, vmalloc) [ 1.326136] UDP hash table entries: 4096 (order: 7, 786432 bytes, vmalloc) [ 1.330057] UDP-Lite hash table entries: 4096 (order: 7, 786432 bytes, vmalloc) [ 1.333735] NET: Registered protocol family 1 [ 1.338154] RPC: Registered named UNIX socket transport module. [ 1.340737] RPC: Registered udp transport module. [ 1.342508] RPC: Registered tcp transport module. [ 1.344473] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.346746] NET: Registered protocol family 44 [ 1.348434] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.350666] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.352860] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.354977] PCI: CLS 0 bytes, default 64 [ 1.357318] Unpacking initramfs... [ 3.489292] debug: unmapping init [mem 0xffff96697cbf3000-0xffff96697ffcffff] [ 3.493497] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.495471] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.498266] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.516324] cryptomgr_test (65) used greatest stack depth: 14248 bytes left [ 5.029689] Initialise system trusted keyrings [ 5.031622] Key type blacklist registered [ 5.035870] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 5.145717] zbud: loaded [ 5.203334] *** VALIDATE nfs *** [ 5.210160] *** VALIDATE nfs4 *** [ 5.221839] pstore: using deflate compression [ 5.235307] Platform Keyring initialized [ 5.253136] cryptomgr_test (73) used greatest stack depth: 14024 bytes left [ 5.315234] cryptomgr_test (86) used greatest stack depth: 13800 bytes left [ 5.445707] cryptomgr_test (94) used greatest stack depth: 13640 bytes left [ 5.449044] cryptomgr_test (90) used greatest stack depth: 13560 bytes left [ 5.732506] NET: Registered protocol family 38 [ 5.736198] Key type asymmetric registered [ 5.739847] Asymmetric key parser 'x509' registered [ 5.741706] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 5.755296] io scheduler mq-deadline registered [ 5.757316] io scheduler kyber registered [ 5.766903] io scheduler bfq registered [ 5.769100] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 5.779923] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 5.790319] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 5.799203] ACPI: Power Button [PWRF] [ 9.186792] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 12.418749] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 15.752959] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 15.859385] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 15.961833] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 16.017757] Non-volatile memory driver v1.3 [ 16.019472] Linux agpgart interface v0.103 [ 16.509507] virtio_blk virtio1: [vda] 131896 512-byte logical blocks (67.5 MB/64.4 MiB) [ 16.534085] vda: detected capacity change from 0 to 67530752 [ 16.626541] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 16.636308] vdb: detected capacity change from 0 to 1073741824 [ 16.715678] libphy: Fixed MDIO Bus: probed [ 16.764844] usbcore: registered new interface driver usbserial_generic [ 16.773056] usbserial: USB Serial support registered for generic [ 16.775655] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 16.800679] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 16.813242] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 16.828147] mousedev: PS/2 mouse device common for all mice [ 16.849244] rtc_cmos 00:05: RTC can wake from S4 [ 16.860975] rtc_cmos 00:05: registered as rtc0 [ 16.863104] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 16.865958] intel_pstate: CPU model not supported [ 16.871429] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 16.909808] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 16.910689] hid: raw HID events driver (C) Jiri Kosina [ 16.932423] usbcore: registered new interface driver usbhid [ 16.934620] usbhid: USB HID core driver [ 16.936367] drop_monitor: Initializing network drop monitor service [ 16.941849] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 16.947615] Initializing XFRM netlink socket [ 16.963893] NET: Registered protocol family 10 [ 16.998674] Segment Routing with IPv6 [ 17.004618] NET: Registered protocol family 17 [ 17.008892] mpls_gso: MPLS GSO support [ 17.035266] RAS: Correctable Errors collector initialized. [ 17.037652] AVX version of gcm_enc/dec engaged. [ 17.051797] AES CTR mode by8 optimization enabled [ 17.720504] sched_clock: Marking stable (17720449039, 0)->(18836775003, -1116325964) [ 17.732556] registered taskstats version 1 [ 17.759248] Loading compiled-in X.509 certificates [ 17.761920] zswap: loaded using pool lzo/zbud [ 17.986694] Key type big_key registered [ 18.086667] Key type encrypted registered [ 18.095627] ima: No TPM chip found, activating TPM-bypass! [ 18.100441] ima: Allocated hash algorithm: sha1 [ 18.102380] ima: No architecture policies found [ 18.104379] evm: Initialising EVM extended attributes: [ 18.106210] evm: security.selinux [ 18.107407] evm: security.ima [ 18.108621] evm: security.capability [ 18.109977] evm: HMAC attrs: 0x1 [ 18.130398] rtc_cmos 00:05: setting system clock to 2025-04-01 08:38:03 UTC (1743496683) [ 18.189832] debug: unmapping init [mem 0xffffffffa4e03000-0xffffffffa4ffffff] [ 18.209251] debug: unmapping init [mem 0xffffffffa3471000-0xffffffffa3902fff] [ 18.218208] Write protecting the kernel read-only data: 30720k [ 18.225550] debug: unmapping init [mem 0xffffffffa1a03000-0xffffffffa1bfffff] [ 18.228652] debug: unmapping init [mem 0xffffffffa232f000-0xffffffffa23fffff] [ 18.539683] systemd[1]: systemd 239 (239-82.el8_10.3) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 18.588043] systemd[1]: Detected virtualization kvm. [ 18.594665] systemd[1]: Detected architecture x86-64. [ 18.612677] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 18.749410] systemd[1]: No hostname configured. [ 18.755581] systemd[1]: Set hostname to . [ 18.761089] random: systemd: uninitialized urandom read (16 bytes read) [ 18.766966] systemd[1]: Initializing machine ID from random generator. [ 19.151722] random: ln: uninitialized urandom read (6 bytes read) [ 19.644476] dracut-nbd-gene (195) used greatest stack depth: 13416 bytes left [ 19.768491] random: systemd: uninitialized urandom read (16 bytes read) [ 19.771254] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 19.789674] random: systemd: uninitialized urandom read (16 bytes read) [ 19.799582] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 19.814661] random: systemd: uninitialized urandom read (16 bytes read) [ 19.818233] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Timers. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Swap. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Create Volatile Files and Directories... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Sockets. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 23.696414] device-mapper: uevent: version 1.0.3 [ 23.703238] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ 23.777398] hrtimer: interrupt took 4112452 ns [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 28.575593] virtio_net virtio0 ens2: renamed from eth0 [ 29.253836] scsi host0: ata_piix [ 29.307059] scsi host1: ata_piix [ 29.309506] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 29.328060] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 32.776712] systemd-udevd (443) used greatest stack depth: 12792 bytes left [ 32.840817] systemd-udevd (446) used greatest stack depth: 12536 bytes left [ 34.372232] ip (530) used greatest stack depth: 11496 bytes left [ 34.681042] random: crng init done [ 34.684822] random: 5 urandom warning(s) missed due to ratelimiting [ 39.329941] dracut-initqueue[592]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 42.141736] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Create Volatile Files and Directories. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 48.327938] printk: systemd: 26 output lines suppressed due to ratelimiting [ 49.837264] SELinux: Disabled at runtime. [ 50.109962] systemd[1]: systemd 239 (239-82.el8_10.3) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 50.129324] systemd[1]: Detected virtualization kvm. [ 50.132694] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 53.128372] systemd[1]: initrd-switch-root.service: Succeeded. [ 53.141045] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 53.190917] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 53.201069] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 53.216276] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 53.368367] systemd[1]: Starting Journal Service... Starting Journal Service... [ 53.435204] systemd[1]: Activating swap /dev/disk/by-label/SWAP... Activating swap /dev/disk/by-label/SWAP... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Switch Root. [ OK ] Listening on udev Control Socket. [ 53.635071] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Kernel Debug File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting POSIX Message Queue File System... [ OK ] Reached target rpc_pipefs.target. Starting Remount Root and Kernel File Systems... [ OK ] Listening on RPCbind Server Activation Socket. Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... Mounting Huge Pages File System... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 57.898645] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 60.142792] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 60.962394] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 62.214598] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 62.472495] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…only root support (11s / no limit) [** ] A start job is running for Configur…only root support (12s / no limit) [*** ] A start job is running for Configur…only root support (12s / no limit) [ *** ] A start job is running for Configur…only root support (13s / no limit) [ *** ] A start job is running for Configur…only root support (13s / no limit) [ ***] A start job is running for Configur…only root support (14s / no limit) [ **] A start job is running for Configur…only root support (14s / no limit) [ *] A start job is running for Configur…only root support (14s / no limit) [ **] A start job is running for Configur…only root support (15s / no limit) [ ***] A start job is running for Configur…only root support (15s / no limit) [ *** ] A start job is running for Configur…only root support (16s / no limit) [ *** ] A start job is running for Configur…only root support (16s / no limit) [*** ] A start job is running for Configur…only root support (17s / no limit) [** ] A start job is running for Configur…only root support (17s / no limit) [* ] A start job is running for Configur…only root support (18s / no limit) [** ] A start job is running for Configur…only root support (19s / no limit) [*** ] A start job is running for Configur…only root support (20s / no limit)[ 73.128923] Key type dns_resolver registered [ *** ] A start job is running for Configur…only root support (20s / no limit) [ *** ] A start job is running for Configur…only root support (21s / no limit) [ ***] A start job is running for Configur…only root support (21s / no limit)[ 74.561362] NFS: Registering the id_resolver key type [ 74.563446] Key type id_resolver registered [ 74.575159] Key type id_legacy registered [ **] A start job is running for Configur…only root support (22s / no limit)[ 75.382096] mount.nfs (966) used greatest stack depth: 10760 bytes left [ *] A start job is running for Configur…only root support (22s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg617-client login: [ 122.937151] mount.nfs (1526) used greatest stack depth: 10456 bytes left [ 213.485027] libcfs: loading out-of-tree module taints kernel. [ 213.543119] Key type ._llcrypt registered [ 213.579151] Key type .llcrypt registered [ 214.586935] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 214.639252] alg: No test for adler32 (adler32-zlib) [ 216.726244] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 218.787461] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 218.791909] LNet: Accept secure, port 988 [ 220.807493] Key type lgssc registered [ 225.213770] Lustre: Echo OBD driver; http://www.lustre.org/ [ 518.929092] Lustre: Mounted lustre-client [ 528.159947] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 544.738966] Lustre: lustre-OST0000-osc-ffff9669c60f4000: disconnect after 23s idle [ 552.708073] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing check_logdir /tmp/testlogs/ [ 560.579106] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing yml_node [ 571.581910] Lustre: DEBUG MARKER: Client: 2.16.52.73 [ 576.423475] Lustre: DEBUG MARKER: MDS: 2.16.52.73 [ 580.572853] Lustre: DEBUG MARKER: OSS: 2.16.52.73 [ 584.245817] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Apr 1 04:47:26 EDT 2025 [ 614.291958] Lustre: DEBUG MARKER: excepting tests: 55 63 67 32newtarball [ 617.372293] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 621.511253] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 631.992486] Lustre: Unmounted lustre-client [ 797.260461] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 803.816491] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 835.035917] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 879.356348] Lustre: DEBUG MARKER: == conf-sanity test 33a: Mount ost with a large index number ========================================================== 04:52:21 (1743497541) [ 883.842035] Lustre: DEBUG MARKER: SKIP: conf-sanity test_33a mixed loopback and real device not working [ 887.714807] Lustre: DEBUG MARKER: == conf-sanity test 33b: Drop cancel during umount ======= 04:52:29 (1743497549) [ 929.931415] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 936.169187] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 960.588164] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 971.880146] Lustre: Mounted lustre-client [ 977.822704] WARNING: CPU: 3 PID: 38 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 977.826587] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [ 977.849533] CPU: 3 PID: 38 Comm: kworker/u8:3 Kdump: loaded Tainted: G O -------- - - 4.18.0rh8.10-debug #7 [ 977.857878] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 977.861435] Workqueue: writeback wb_workfn (flush-lustre-ffff9669c8) [ 977.868211] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 977.882449] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 977.889992] RSP: 0018:ffffa71ec0dabb58 EFLAGS: 00010202 [ 977.892648] RAX: 0000000000000000 RBX: ffffa71ec0dabc40 RCX: 0000000000000001 [ 977.895769] RDX: 0000000000000007 RSI: ffff9669c61b10c8 RDI: 0000000000000202 [ 977.898240] RBP: ffff9669c6f12210 R08: 0000000000000000 R09: 0000000000000001 [ 977.900620] R10: 0000000000331323 R11: ffff9669c9979000 R12: ffff9669c6f12450 [ 977.903545] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 977.906276] FS: 0000000000000000(0000) GS:ffff966a01e00000(0000) knlGS:0000000000000000 [ 977.909430] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 977.911893] CR2: 000055f9c5873028 CR3: 0000000024228005 CR4: 0000000000170ee0 [ 977.915139] Call Trace: [ 977.916144] ? show_regs.cold.9+0x22/0x2f [ 977.917727] ? __warn+0xc8/0x150 [ 977.919098] ? ll_writepages+0x3dd/0x400 [lustre] [ 977.921082] ? report_bug+0x113/0x140 [ 977.922523] ? do_error_trap+0xb6/0x130 [ 977.924344] ? do_invalid_op+0x46/0x60 [ 977.925980] ? ll_writepages+0x3dd/0x400 [lustre] [ 977.928204] ? invalid_op+0x14/0x20 [ 977.929862] ? ll_writepages+0x3dd/0x400 [lustre] [ 977.932292] ? ll_writepages+0x3b8/0x400 [lustre] [ 977.934504] do_writepages+0x7a/0x270 [ 977.935946] __writeback_single_inode+0xb2/0x7d0 [ 977.937821] ? do_raw_spin_unlock+0x75/0x190 [ 977.939548] writeback_sb_inodes+0x239/0x7e0 [ 977.941296] wb_writeback+0x1ae/0x7a0 [ 977.942777] wb_workfn+0x1e5/0x950 [ 977.944240] ? lock_acquire+0x146/0x3a0 [ 977.945704] ? process_one_work+0x2c0/0x9b0 [ 977.947421] process_one_work+0x3dd/0x9b0 [ 977.949101] worker_thread+0x296/0x6e0 [ 977.950624] ? rescuer_thread+0x570/0x570 [ 977.952280] kthread+0x1d7/0x210 [ 977.953588] ? set_kthread_struct+0x70/0x70 [ 977.955538] ret_from_fork+0x1f/0x30 [ 977.957072] ---[ end trace 4a77805e2a5b90b0 ]--- [ 978.295801] Lustre: Unmounted lustre-client [ 1015.597276] Key type lgssc unregistered [ 1016.323833] LNet: 14387:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1017.412380] LNet: Removed LNI 192.168.206.17@tcp [ 1019.339599] Key type .llcrypt unregistered [ 1019.341404] Key type ._llcrypt unregistered [ 1047.992155] Key type ._llcrypt registered [ 1047.993811] Key type .llcrypt registered [ 1050.777719] Lustre: DEBUG MARKER: == conf-sanity test 33c: Mount ost with a large index number ========================================================== 04:55:12 (1743497712) [ 1055.598467] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1055.660779] alg: No test for adler32 (adler32-zlib) [ 1057.182211] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 1057.651526] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 1057.655905] LNet: Accept secure, port 988 [ 1059.487394] Key type lgssc registered [ 1061.798825] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1153.389077] Lustre: Mounted lustre-client [ 1244.693637] Lustre: Unmounted lustre-client [ 1283.696975] Key type lgssc unregistered [ 1284.313272] LNet: 18039:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1285.359771] LNet: Removed LNI 192.168.206.17@tcp [ 1287.299623] Key type .llcrypt unregistered [ 1287.304510] Key type ._llcrypt unregistered [ 1307.011616] Key type ._llcrypt registered [ 1307.013632] Key type .llcrypt registered [ 1309.572188] Lustre: DEBUG MARKER: == conf-sanity test 33d: Don't panic when enable project quota ========================================================== 04:59:32 (1743497972) [ 1314.906757] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1314.945244] alg: No test for adler32 (adler32-zlib) [ 1316.351885] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 1316.811212] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 1316.819067] LNet: Accept secure, port 988 [ 1318.703434] Key type lgssc registered [ 1321.189266] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1382.820212] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1388.941957] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1411.919634] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1413.878384] Lustre: Mounted lustre-client [ 1430.255777] WARNING: CPU: 1 PID: 12434 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 1430.261008] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 1430.282495] CPU: 1 PID: 12434 Comm: kworker/u8:1 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 1430.289968] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 1430.292942] Workqueue: writeback wb_workfn (flush-lustre-ffff966a00) [ 1430.295321] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 1430.297327] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 1430.303508] RSP: 0018:ffffa71ec2023b58 EFLAGS: 00010202 [ 1430.305350] RAX: 0000000000000000 RBX: ffffa71ec2023c40 RCX: 0000000000000001 [ 1430.307764] RDX: 0000000000000007 RSI: ffff9669f9c6c0c8 RDI: 0000000000000202 [ 1430.310103] RBP: ffff9669d818c290 R08: 0000000000000000 R09: 0000000000000001 [ 1430.312419] R10: 000000000282fefd R11: ffff966a00777000 R12: ffff9669d818c4d0 [ 1430.314942] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 1430.317165] FS: 0000000000000000(0000) GS:ffff966a01600000(0000) knlGS:0000000000000000 [ 1430.319833] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1430.321763] CR2: 000056085cfa6f44 CR3: 0000000024228001 CR4: 0000000000170ee0 [ 1430.324193] Call Trace: [ 1430.325129] ? show_regs.cold.9+0x22/0x2f [ 1430.326496] ? __warn+0xc8/0x150 [ 1430.327606] ? ll_writepages+0x3dd/0x400 [lustre] [ 1430.329459] ? report_bug+0x113/0x140 [ 1430.330755] ? do_error_trap+0xb6/0x130 [ 1430.332225] ? do_invalid_op+0x46/0x60 [ 1430.333658] ? ll_writepages+0x3dd/0x400 [lustre] [ 1430.335428] ? invalid_op+0x14/0x20 [ 1430.336714] ? ll_writepages+0x3dd/0x400 [lustre] [ 1430.338388] ? ll_writepages+0x3b8/0x400 [lustre] [ 1430.340116] do_writepages+0x7a/0x270 [ 1430.341225] __writeback_single_inode+0xb2/0x7d0 [ 1430.342911] ? do_raw_spin_unlock+0x75/0x190 [ 1430.344448] writeback_sb_inodes+0x239/0x7e0 [ 1430.345868] wb_writeback+0x1ae/0x7a0 [ 1430.347153] wb_workfn+0x1e5/0x950 [ 1430.348442] ? lock_acquire+0x146/0x3a0 [ 1430.349886] ? process_one_work+0x2c0/0x9b0 [ 1430.351393] process_one_work+0x3dd/0x9b0 [ 1430.352950] worker_thread+0x296/0x6e0 [ 1430.354367] ? rescuer_thread+0x570/0x570 [ 1430.355861] kthread+0x1d7/0x210 [ 1430.357089] ? set_kthread_struct+0x70/0x70 [ 1430.358698] ret_from_fork+0x1f/0x30 [ 1430.360070] ---[ end trace 4a77805e2a5b90b1 ]--- [ 1430.476651] Lustre: Unmounted lustre-client [ 1472.917810] Key type lgssc unregistered [ 1473.589988] LNet: 21418:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1474.670320] LNet: Removed LNI 192.168.206.17@tcp [ 1476.352956] Key type .llcrypt unregistered [ 1476.355261] Key type ._llcrypt unregistered [ 1495.801645] Key type ._llcrypt registered [ 1495.804922] Key type .llcrypt registered [ 1498.371318] Lustre: DEBUG MARKER: == conf-sanity test 34a: umount with opened file should be fail ========================================================== 05:02:40 (1743498160) [ 1502.819717] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1502.865096] alg: No test for adler32 (adler32-zlib) [ 1504.254275] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 1504.740584] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 1504.746536] LNet: Accept secure, port 988 [ 1506.535385] Key type lgssc registered [ 1508.956690] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1567.952497] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1573.352916] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1596.442803] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1598.334402] Lustre: Mounted lustre-client [ 1607.531891] Lustre: Unmounted lustre-client [ 1646.963941] Key type lgssc unregistered [ 1647.525955] LNet: 24375:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1648.615262] LNet: Removed LNI 192.168.206.17@tcp [ 1650.080371] Key type .llcrypt unregistered [ 1650.082192] Key type ._llcrypt unregistered [ 1676.472697] Key type ._llcrypt registered [ 1676.477197] Key type .llcrypt registered [ 1679.150187] Lustre: DEBUG MARKER: == conf-sanity test 34b: force umount with failed mds should be normal ========================================================== 05:05:41 (1743498341) [ 1684.471105] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1684.511176] alg: No test for adler32 (adler32-zlib) [ 1685.841295] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 1686.431747] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 1686.434357] LNet: Accept secure, port 988 [ 1688.327321] Key type lgssc registered [ 1690.622828] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1749.547726] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1755.187174] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1776.869320] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1778.915857] Lustre: Mounted lustre-client [ 1789.434499] Lustre: lustre-MDT0000-mdc-ffff9669d0049000: Connection to lustre-MDT0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 1799.660877] Lustre: lustre-MDT0001-mdc-ffff9669d0049000: Connection to lustre-MDT0001 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 1805.576924] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 1809.888529] Lustre: 25321:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743498459/real 1743498459] req@0000000045e11ba1 x1828190526904320/t0(0) o400->MGC192.168.206.117@tcp@192.168.206.117@tcp:26/25 lens 224/224 e 0 to 1 dl 1743498475 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 1809.911526] LustreError: MGC192.168.206.117@tcp: Connection to MGS (at 192.168.206.117@tcp) was lost; in progress operations using this service will fail [ 1815.519606] WARNING: CPU: 0 PID: 38 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 1815.524204] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 1815.550212] CPU: 0 PID: 38 Comm: kworker/u8:3 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 1815.554129] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 1815.557511] Workqueue: writeback wb_workfn (flush-lustre-ffff9669d0) [ 1815.559868] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 1815.562338] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 1815.578055] RSP: 0018:ffffa71ec0dabb18 EFLAGS: 00010202 [ 1815.586043] RAX: 0000000000000000 RBX: ffffa71ec0dabc00 RCX: 0000000000000001 [ 1815.591496] RDX: 000000000000000e RSI: ffff9669f16d40c8 RDI: 0000000000000206 [ 1815.598786] RBP: ffff9669d8226310 R08: 0000000000000000 R09: 0000000000000001 [ 1815.605444] R10: 00000000029fa1a1 R11: ffff9669f1537000 R12: ffff9669d8226550 [ 1815.607851] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000 [ 1815.609882] FS: 0000000000000000(0000) GS:ffff966a01200000(0000) knlGS:0000000000000000 [ 1815.618282] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1815.621757] CR2: 00007f37af7a8070 CR3: 0000000024228006 CR4: 0000000000170ef0 [ 1815.625276] Call Trace: [ 1815.626442] ? show_regs.cold.9+0x22/0x2f [ 1815.627970] ? __warn+0xc8/0x150 [ 1815.629659] ? ll_writepages+0x3dd/0x400 [lustre] [ 1815.632739] ? report_bug+0x113/0x140 [ 1815.634154] ? do_error_trap+0xb6/0x130 [ 1815.635696] ? do_invalid_op+0x46/0x60 [ 1815.637896] ? ll_writepages+0x3dd/0x400 [lustre] [ 1815.640295] ? invalid_op+0x14/0x20 [ 1815.644731] ? ll_writepages+0x3dd/0x400 [lustre] [ 1815.646744] ? ll_writepages+0x3b8/0x400 [lustre] [ 1815.650297] do_writepages+0x7a/0x270 [ 1815.652378] __writeback_single_inode+0xb2/0x7d0 [ 1815.654887] ? do_raw_spin_unlock+0x75/0x190 [ 1815.656948] writeback_sb_inodes+0x239/0x7e0 [ 1815.658844] __writeback_inodes_wb+0x71/0x110 [ 1815.661069] wb_writeback+0x627/0x7a0 [ 1815.662400] ? get_nr_inodes+0x45/0x70 [ 1815.664442] wb_workfn+0x778/0x950 [ 1815.666921] ? sched_clock_cpu+0x24/0x150 [ 1815.668389] process_one_work+0x3dd/0x9b0 [ 1815.669664] worker_thread+0x296/0x6e0 [ 1815.670875] ? rescuer_thread+0x570/0x570 [ 1815.671834] kthread+0x1d7/0x210 [ 1815.672719] ? set_kthread_struct+0x70/0x70 [ 1815.673775] ret_from_fork+0x1f/0x30 [ 1815.674796] ---[ end trace 4a77805e2a5b90b2 ]--- [ 1815.934555] Lustre: Unmounted lustre-client [ 1833.977713] Key type lgssc unregistered [ 1834.671452] LNet: 27804:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1835.761426] LNet: Removed LNI 192.168.206.17@tcp [ 1837.398723] Key type .llcrypt unregistered [ 1837.400621] Key type ._llcrypt unregistered [ 1863.508738] Key type ._llcrypt registered [ 1863.513875] Key type .llcrypt registered [ 1866.364721] Lustre: DEBUG MARKER: == conf-sanity test 34c: force umount with failed ost should be normal ========================================================== 05:08:48 (1743498528) [ 1871.789909] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1871.818534] alg: No test for adler32 (adler32-zlib) [ 1873.143742] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 1873.527243] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 1873.530587] LNet: Accept secure, port 988 [ 1875.383354] Key type lgssc registered [ 1877.599720] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1938.686873] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1944.938517] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1966.681876] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1968.705899] Lustre: Mounted lustre-client [ 1978.868878] Lustre: lustre-OST0000-osc-ffff9669f164c000: Connection to lustre-OST0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 1984.588224] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 1994.853521] WARNING: CPU: 2 PID: 12434 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 1994.862468] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 1994.887253] CPU: 2 PID: 12434 Comm: kworker/u8:1 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 1994.892231] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 1994.895176] Workqueue: writeback wb_workfn (flush-lustre-ffff9669f1) [ 1994.897794] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 1994.900726] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 1994.906559] RSP: 0018:ffffa71ec2023b58 EFLAGS: 00010202 [ 1994.908355] RAX: 0000000000000000 RBX: ffffa71ec2023c40 RCX: 0000000000000001 [ 1994.911350] RDX: 0000000000000007 RSI: ffff9669eb2d90c8 RDI: 0000000000000202 [ 1994.913254] RBP: ffff9669d8270190 R08: 0000000000000000 R09: 0000000000000001 [ 1994.915297] R10: 0000000002ad6ac9 R11: ffff9669fa137000 R12: ffff9669d82703d0 [ 1994.918059] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 1994.920304] FS: 0000000000000000(0000) GS:ffff966a01a00000(0000) knlGS:0000000000000000 [ 1994.922484] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1994.924123] CR2: 00007fe89cf42f20 CR3: 0000000024228006 CR4: 0000000000170ee0 [ 1994.926373] Call Trace: [ 1994.927210] ? show_regs.cold.9+0x22/0x2f [ 1994.928386] ? __warn+0xc8/0x150 [ 1994.929229] ? ll_writepages+0x3dd/0x400 [lustre] [ 1994.930562] ? report_bug+0x113/0x140 [ 1994.931807] ? do_error_trap+0xb6/0x130 [ 1994.937232] ? do_invalid_op+0x46/0x60 [ 1994.938329] ? ll_writepages+0x3dd/0x400 [lustre] [ 1994.941594] ? invalid_op+0x14/0x20 [ 1994.943880] ? ll_writepages+0x3dd/0x400 [lustre] [ 1994.949290] ? ll_writepages+0x3b8/0x400 [lustre] [ 1994.953248] do_writepages+0x7a/0x270 [ 1994.957775] __writeback_single_inode+0xb2/0x7d0 [ 1994.959677] ? do_raw_spin_unlock+0x75/0x190 [ 1994.964054] writeback_sb_inodes+0x239/0x7e0 [ 1994.968956] wb_writeback+0x1ae/0x7a0 [ 1994.969961] wb_workfn+0x1e5/0x950 [ 1994.971178] ? lock_acquire+0x146/0x3a0 [ 1994.972387] ? process_one_work+0x2c0/0x9b0 [ 1994.974050] process_one_work+0x3dd/0x9b0 [ 1994.976290] worker_thread+0x296/0x6e0 [ 1994.979959] ? rescuer_thread+0x570/0x570 [ 1994.983605] kthread+0x1d7/0x210 [ 1994.984868] ? set_kthread_struct+0x70/0x70 [ 1994.986660] ret_from_fork+0x1f/0x30 [ 1994.988011] ---[ end trace 4a77805e2a5b90b3 ]--- [ 1995.074814] Lustre: Unmounted lustre-client [ 2027.823572] Key type lgssc unregistered [ 2028.428390] LNet: 31184:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2029.486684] LNet: Removed LNI 192.168.206.17@tcp [ 2031.275464] Key type .llcrypt unregistered [ 2031.276790] Key type ._llcrypt unregistered [ 2057.213900] Key type ._llcrypt registered [ 2057.215557] Key type .llcrypt registered [ 2059.409509] Lustre: DEBUG MARKER: == conf-sanity test 35a: Reconnect to the last active server first ========================================================== 05:12:02 (1743498722) [ 2064.120352] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2064.137283] alg: No test for adler32 (adler32-zlib) [ 2065.458274] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 2065.870561] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 2065.873433] LNet: Accept secure, port 988 [ 2067.663468] Key type lgssc registered [ 2069.714187] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2123.702612] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2128.707777] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2149.400304] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2151.373244] Lustre: Mounted lustre-client [ 2156.538213] Lustre: DEBUG MARKER: Set up a fake failnode for the MDS [ 2161.344151] Lustre: DEBUG MARKER: Wait for RECONNECT_INTERVAL seconds (10s) [ 2163.211084] LNetError: Refusing connection from 192.168.206.17 for 127.0.0.2@tcp: No matching NI [ 2163.226828] LNetError: 32120:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 2163.231280] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 2173.630362] Lustre: DEBUG MARKER: conf-sanity.sh test_35a 2025-04-01 5h13m56s [ 2176.401562] Lustre: DEBUG MARKER: Stopping the MDT: lustre-MDT0000 [ 2182.125136] Lustre: lustre-MDT0000-mdc-ffff9669c7904000: Connection to lustre-MDT0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 2182.147655] LNetError: Refusing connection from 192.168.206.17 for 127.0.0.2@tcp: No matching NI [ 2182.153258] LNetError: 32121:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 2182.160792] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 2186.758135] LustreError: 34741:0:(lmv_obd.c:1435:lmv_statfs()) lustre-MDT0000-mdc-ffff9669c7904000: can't stat MDS #0: rc = -19 [ 2188.258239] LNetError: Refusing connection from 192.168.206.17 for 127.0.0.2@tcp: No matching NI [ 2188.263863] LNetError: 32122:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 2188.267750] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 2189.045177] Lustre: DEBUG MARKER: Restarting the MDT: lustre-MDT0000 [ 2194.401811] LNetError: Refusing connection from 192.168.206.17 for 127.0.0.2@tcp: No matching NI [ 2194.408964] LNetError: 32119:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 2194.415653] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 2200.550148] LustreError: MGC192.168.206.117@tcp: Connection to MGS (at 192.168.206.117@tcp) was lost; in progress operations using this service will fail [ 2200.564810] Lustre: Evicted from MGS (at 192.168.206.117@tcp) after server handle changed from 0x79cca267c1e6e5d7 to 0x79cca267c1e6e88c [ 2200.573996] Lustre: MGC192.168.206.117@tcp: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 2201.580214] LustreError: lustre-MDT0000-mdc-ffff9669c7904000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. [ 2201.594439] Lustre: lustre-MDT0000-mdc-ffff9669c7904000: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 2204.870250] Lustre: DEBUG MARKER: Wait for df (34741) ... [ 2207.417504] Lustre: DEBUG MARKER: done [ 2209.416096] Lustre: Unmounted lustre-client [ 2248.315856] Key type lgssc unregistered [ 2248.853234] LNet: 35745:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2249.895874] LNet: Removed LNI 192.168.206.17@tcp [ 2251.273971] Key type .llcrypt unregistered [ 2251.275073] Key type ._llcrypt unregistered [ 2301.352964] Key type ._llcrypt registered [ 2301.354649] Key type .llcrypt registered [ 2302.026988] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2302.057616] alg: No test for adler32 (adler32-zlib) [ 2303.322130] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 2303.720077] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 2303.723437] LNet: Accept secure, port 988 [ 2305.519595] Key type lgssc registered [ 2307.629576] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2400.209777] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2405.465800] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2429.580371] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2475.928309] Lustre: DEBUG MARKER: == conf-sanity test 35b: Continue reconnection retries, if the active server is busy ========================================================== 05:18:58 (1743499138) [ 2508.658687] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2513.778605] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2532.033852] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2533.801519] Lustre: Mounted lustre-client [ 2539.291062] Lustre: DEBUG MARKER: conf-sanity.sh test_35b 2025-04-01 5h20m02s [ 2541.628276] Lustre: DEBUG MARKER: Set up a fake failnode for the MDS [ 2549.824801] Lustre: DEBUG MARKER: Injecting EBUSY on MDS [ 2550.281543] LNetError: Refusing connection from 192.168.206.17 for 127.0.0.2@tcp: No matching NI [ 2550.287980] LNetError: 36521:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 2550.295505] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 2552.928372] Lustre: DEBUG MARKER: Creating a test file and stat it [ 2573.279192] Lustre: 42412:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743499218/real 1743499218] req@0000000043dc723c x1828191174924928/t0(0) o101->lustre-MDT0000-mdc-ffff9669f0a79000@192.168.206.117@tcp:12/10 lens 576/1152 e 0 to 1 dl 1743499238 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 2573.290878] Lustre: lustre-MDT0000-mdc-ffff9669f0a79000: Connection to lustre-MDT0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 2573.325223] Lustre: lustre-MDT0000-mdc-ffff9669f0a79000: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 2595.222339] Lustre: DEBUG MARKER: Stop injecting EBUSY on MDS [ 2598.607824] Lustre: DEBUG MARKER: done [ 2602.446459] Lustre: Unmounted lustre-client [ 2630.716478] Key type lgssc unregistered [ 2631.256245] LNet: 43295:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2632.301191] LNet: Removed LNI 192.168.206.17@tcp [ 2633.516536] Key type .llcrypt unregistered [ 2633.518631] Key type ._llcrypt unregistered [ 2682.064381] Key type ._llcrypt registered [ 2682.067512] Key type .llcrypt registered [ 2682.444272] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2682.468877] alg: No test for adler32 (adler32-zlib) [ 2683.740310] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 2684.118130] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 2684.122457] LNet: Accept secure, port 988 [ 2685.879347] Key type lgssc registered [ 2687.919497] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2765.026923] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2769.549760] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2789.541287] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2845.302701] Lustre: DEBUG MARKER: == conf-sanity test 36: df report consistency on OSTs with different block size ========================================================== 05:25:08 (1743499508) [ 2847.536812] Lustre: DEBUG MARKER: SKIP: conf-sanity test_36 remote OST [ 2849.772381] Lustre: DEBUG MARKER: == conf-sanity test 37: verify set tunables works for symlink device ========================================================== 05:25:13 (1743499513) [ 2880.834889] Lustre: DEBUG MARKER: == conf-sanity test 38: MDS recreates missing lov_objid file from OST data ========================================================== 05:25:44 (1743499544) [ 2907.324601] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2911.691539] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2928.657712] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2934.801410] Lustre: Mounted lustre-client [ 2939.164193] Lustre: DEBUG MARKER: copying 10 files to /mnt/lustre/d38.conf-sanity [ 2941.105550] WARNING: CPU: 2 PID: 12434 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 2941.115303] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 2941.134500] CPU: 2 PID: 12434 Comm: kworker/u8:1 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 2941.139013] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 2941.142444] Workqueue: writeback wb_workfn (flush-lustre-ffff9669f9) [ 2941.144857] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 2941.146742] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 2941.153925] RSP: 0018:ffffa71ec2023b18 EFLAGS: 00010202 [ 2941.156448] RAX: 0000000000000000 RBX: ffffa71ec2023c00 RCX: 0000000000000001 [ 2941.165015] RDX: 000000000000000e RSI: ffff966a00dbc0c8 RDI: 0000000000000206 [ 2941.168001] RBP: ffff9669eb79a210 R08: 0000000000000000 R09: 0000000000000001 [ 2941.172464] R10: 0000000002ef5610 R11: ffff9669eaa22000 R12: ffff9669eb79a450 [ 2941.175461] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000 [ 2941.177970] FS: 0000000000000000(0000) GS:ffff966a01a00000(0000) knlGS:0000000000000000 [ 2941.187771] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2941.189633] CR2: 000055d8923da1b8 CR3: 0000000024228002 CR4: 0000000000170ee0 [ 2941.192613] Call Trace: [ 2941.193624] ? show_regs.cold.9+0x22/0x2f [ 2941.195351] ? __warn+0xc8/0x150 [ 2941.196691] ? ll_writepages+0x3dd/0x400 [lustre] [ 2941.199168] ? report_bug+0x113/0x140 [ 2941.200700] ? do_error_trap+0xb6/0x130 [ 2941.202312] ? do_invalid_op+0x46/0x60 [ 2941.204272] ? ll_writepages+0x3dd/0x400 [lustre] [ 2941.205969] ? invalid_op+0x14/0x20 [ 2941.207221] ? ll_writepages+0x3dd/0x400 [lustre] [ 2941.208809] ? ll_writepages+0x3b8/0x400 [lustre] [ 2941.210053] do_writepages+0x7a/0x270 [ 2941.211361] __writeback_single_inode+0xb2/0x7d0 [ 2941.213121] ? do_raw_spin_unlock+0x75/0x190 [ 2941.214652] writeback_sb_inodes+0x239/0x7e0 [ 2941.216384] __writeback_inodes_wb+0x71/0x110 [ 2941.219891] wb_writeback+0x627/0x7a0 [ 2941.221250] ? get_nr_inodes+0x45/0x70 [ 2941.222381] wb_workfn+0x30b/0x950 [ 2941.223574] ? sched_clock_cpu+0x24/0x150 [ 2941.224706] process_one_work+0x3dd/0x9b0 [ 2941.227473] worker_thread+0x296/0x6e0 [ 2941.231053] ? rescuer_thread+0x570/0x570 [ 2941.232529] kthread+0x1d7/0x210 [ 2941.235054] ? set_kthread_struct+0x70/0x70 [ 2941.236793] ret_from_fork+0x1f/0x30 [ 2941.237964] ---[ end trace 4a77805e2a5b90b4 ]--- [ 2942.972793] Lustre: Unmounted lustre-client [ 2953.196633] Lustre: DEBUG MARKER: delete lov_objid file on MDS [ 2997.641346] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3002.286404] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3003.169221] Lustre: Mounted lustre-client [ 3006.789616] Lustre: Unmounted lustre-client [ 3058.679278] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3062.805118] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3063.616459] Lustre: Mounted lustre-client [ 3067.061103] Lustre: Unmounted lustre-client [ 3084.825664] Lustre: DEBUG MARKER: files compared the same [ 3109.636876] Key type lgssc unregistered [ 3110.115463] LNet: 53861:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3111.145450] LNet: Removed LNI 192.168.206.17@tcp [ 3112.481924] Key type .llcrypt unregistered [ 3112.483524] Key type ._llcrypt unregistered [ 3133.055699] Key type ._llcrypt registered [ 3133.057258] Key type .llcrypt registered [ 3134.886465] Lustre: DEBUG MARKER: == conf-sanity test 39: leak_finder recognizes both LUSTRE and LNET malloc messages ========================================================== 05:29:58 (1743499798) [ 3138.483852] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3138.507322] alg: No test for adler32 (adler32-zlib) [ 3139.718117] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 3140.065238] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 3140.068812] LNet: Accept secure, port 988 [ 3141.880130] Key type lgssc registered [ 3143.728762] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3190.963987] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3195.366689] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3212.774810] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3214.605699] Lustre: Mounted lustre-client [ 3218.760848] Lustre: Unmounted lustre-client [ 3256.597372] Key type lgssc unregistered [ 3257.128627] LNet: 57203:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3258.151088] LNet: Removed LNI 192.168.206.17@tcp [ 3259.375229] Key type .llcrypt unregistered [ 3259.376959] Key type ._llcrypt unregistered [ 3281.728769] Key type ._llcrypt registered [ 3281.730279] Key type .llcrypt registered [ 3283.762481] Lustre: DEBUG MARKER: == conf-sanity test 40: race during service thread startup ========================================================== 05:32:26 (1743499946) [ 3288.171791] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3288.190380] alg: No test for adler32 (adler32-zlib) [ 3289.470076] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 3289.838945] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 3289.842288] LNet: Accept secure, port 988 [ 3291.631335] Key type lgssc registered [ 3293.697355] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3382.865877] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3410.352360] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3415.227872] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3445.498378] Key type lgssc unregistered [ 3446.008371] LNet: 60446:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3446.020518] LNet: Removed LNI 192.168.206.17@tcp [ 3447.227933] Key type .llcrypt unregistered [ 3447.229772] Key type ._llcrypt unregistered [ 3467.407777] Key type ._llcrypt registered [ 3467.409387] Key type .llcrypt registered [ 3469.244681] Lustre: DEBUG MARKER: == conf-sanity test 41a: mount mds with --nosvc and --nomgs ========================================================== 05:35:32 (1743500132) [ 3473.839496] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3473.862641] alg: No test for adler32 (adler32-zlib) [ 3475.128217] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 3475.506366] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 3475.511177] LNet: Accept secure, port 988 [ 3477.256404] Key type lgssc registered [ 3479.136431] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3546.753835] Lustre: Mounted lustre-client [ 3553.459735] WARNING: CPU: 1 PID: 62522 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 3553.468773] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 3553.493697] CPU: 1 PID: 62522 Comm: kworker/u8:2 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 3553.499467] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 3553.503064] Workqueue: writeback wb_workfn (flush-lustre-ffff9669ed) [ 3553.506094] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 3553.508302] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 3553.515433] RSP: 0018:ffffa71ec8febb58 EFLAGS: 00010202 [ 3553.518018] RAX: 0000000000000000 RBX: ffffa71ec8febc40 RCX: 0000000000000001 [ 3553.520300] RDX: 0000000000000007 RSI: ffff9669f95d10c8 RDI: 0000000000000206 [ 3553.522697] RBP: ffff9669eb790190 R08: 0000000000000000 R09: 0000000000000001 [ 3553.525061] R10: 000000000353c4a0 R11: ffff966a00bf9000 R12: ffff9669eb7903d0 [ 3553.527524] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 3553.530331] FS: 0000000000000000(0000) GS:ffff966a01600000(0000) knlGS:0000000000000000 [ 3553.533154] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3553.535263] CR2: 000056229f1b66a8 CR3: 0000000024228005 CR4: 0000000000170ee0 [ 3553.537728] Call Trace: [ 3553.538863] ? show_regs.cold.9+0x22/0x2f [ 3553.540322] ? __warn+0xc8/0x150 [ 3553.545069] ? ll_writepages+0x3dd/0x400 [lustre] [ 3553.546823] ? report_bug+0x113/0x140 [ 3553.548006] ? do_error_trap+0xb6/0x130 [ 3553.549242] ? do_invalid_op+0x46/0x60 [ 3553.554468] ? ll_writepages+0x3dd/0x400 [lustre] [ 3553.556461] ? invalid_op+0x14/0x20 [ 3553.557862] ? ll_writepages+0x3dd/0x400 [lustre] [ 3553.560457] ? ll_writepages+0x3b8/0x400 [lustre] [ 3553.562374] do_writepages+0x7a/0x270 [ 3553.563662] __writeback_single_inode+0xb2/0x7d0 [ 3553.565369] ? do_raw_spin_unlock+0x75/0x190 [ 3553.566952] writeback_sb_inodes+0x239/0x7e0 [ 3553.568652] wb_writeback+0x1ae/0x7a0 [ 3553.569811] wb_workfn+0x1e5/0x950 [ 3553.570969] ? lock_acquire+0x146/0x3a0 [ 3553.572288] ? process_one_work+0x2c0/0x9b0 [ 3553.573461] process_one_work+0x3dd/0x9b0 [ 3553.574741] worker_thread+0x296/0x6e0 [ 3553.575966] ? rescuer_thread+0x570/0x570 [ 3553.577394] kthread+0x1d7/0x210 [ 3553.578548] ? set_kthread_struct+0x70/0x70 [ 3553.580086] ret_from_fork+0x1f/0x30 [ 3553.581166] ---[ end trace 4a77805e2a5b90b5 ]--- [ 3553.697136] Lustre: Unmounted lustre-client [ 3592.067829] Lustre: DEBUG MARKER: == conf-sanity test 41b: mount mds with --nosvc and --nomgs on first mount ========================================================== 05:37:35 (1743500255) [ 3708.206943] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3727.884440] Lustre: Mounted lustre-client [ 3734.482797] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 3734.487123] LustreError: 66549:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff9669fa019000: inode [0x200000402:0x1:0x0] mdc close failed: rc = -108 [ 3734.498197] Lustre: 61415:0:(llite_lib.c:4142:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.206.117@tcp:/lustre/fid: [0x200000402:0x1:0x0]/ may get corrupted (rc -108) [ 3744.816902] Lustre: Unmounted lustre-client [ 3796.484441] Lustre: DEBUG MARKER: == conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one ========================================================== 05:40:59 (1743500459) [ 3809.456582] Key type lgssc unregistered [ 3809.946804] LNet: 67762:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3810.982941] LNet: Removed LNI 192.168.206.17@tcp [ 3812.213846] Key type .llcrypt unregistered [ 3812.215326] Key type ._llcrypt unregistered [ 3829.151794] Key type ._llcrypt registered [ 3829.153494] Key type .llcrypt registered [ 3829.513087] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 3829.533846] alg: No test for adler32 (adler32-zlib) [ 3830.731301] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 3831.050198] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 3831.052879] LNet: Accept secure, port 988 [ 3832.799430] Key type lgssc registered [ 3834.633744] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3876.943036] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3881.005360] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3897.153690] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3908.097525] Lustre: Mounted lustre-client [ 3911.651601] Lustre: Unmounted lustre-client [ 3939.722918] Key type lgssc unregistered [ 3940.207376] LNet: 70532:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3941.227657] LNet: Removed LNI 192.168.206.17@tcp [ 3942.514424] Key type .llcrypt unregistered [ 3942.516074] Key type ._llcrypt unregistered [ 3958.423196] Key type ._llcrypt registered [ 3958.424741] Key type .llcrypt registered [ 3958.945163] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 3958.967252] alg: No test for adler32 (adler32-zlib) [ 3960.189636] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 3960.585662] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 3960.589252] LNet: Accept secure, port 988 [ 3962.327344] Key type lgssc registered [ 3964.204254] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4055.361135] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4059.625986] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4075.591795] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4081.686206] Lustre: Mounted lustre-client [ 4083.573932] Lustre: Unmounted lustre-client [ 4106.699684] Key type lgssc unregistered [ 4107.180717] LNet: 73703:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4108.200665] LNet: Removed LNI 192.168.206.17@tcp [ 4109.425317] Key type .llcrypt unregistered [ 4109.426859] Key type ._llcrypt unregistered [ 4129.206491] Key type ._llcrypt registered [ 4129.208178] Key type .llcrypt registered [ 4130.908788] Lustre: DEBUG MARKER: == conf-sanity test 42: allow client/server mount/unmount with invalid config param ========================================================== 05:46:34 (1743500794) [ 4135.155154] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4135.177692] alg: No test for adler32 (adler32-zlib) [ 4136.447976] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 4136.804419] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 4136.810935] LNet: Accept secure, port 988 [ 4138.647440] Key type lgssc registered [ 4140.811666] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4185.270319] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4189.746145] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4207.040251] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4213.291687] Lustre: Mounted lustre-client [ 4218.271709] Lustre: Unmounted lustre-client [ 4218.824114] Lustre: Mounted lustre-client [ 4221.635756] Lustre: Unmounted lustre-client [ 4250.366890] Key type lgssc unregistered [ 4250.861572] LNet: 77150:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4251.884838] LNet: Removed LNI 192.168.206.17@tcp [ 4253.184707] Key type .llcrypt unregistered [ 4253.186738] Key type ._llcrypt unregistered [ 4269.797595] Key type ._llcrypt registered [ 4269.801411] Key type .llcrypt registered [ 4270.643272] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 4270.668028] alg: No test for adler32 (adler32-zlib) [ 4271.870391] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 4272.231883] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 4272.235261] LNet: Accept secure, port 988 [ 4273.999372] Key type lgssc registered [ 4275.718402] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4321.204564] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4325.883552] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4344.065452] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4350.493701] Lustre: Mounted lustre-client [ 4354.914983] Lustre: Unmounted lustre-client [ 4384.649783] Key type lgssc unregistered [ 4385.199626] LNet: 79941:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4386.216074] LNet: Removed LNI 192.168.206.17@tcp [ 4387.681464] Key type .llcrypt unregistered [ 4387.683461] Key type ._llcrypt unregistered [ 4408.125569] Key type ._llcrypt registered [ 4408.127426] Key type .llcrypt registered [ 4410.056394] Lustre: DEBUG MARKER: == conf-sanity test 43a: check root_squash and nosquash_nids ========================================================== 05:51:13 (1743501073) [ 4413.974924] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4413.999770] alg: No test for adler32 (adler32-zlib) [ 4415.260317] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 4415.624745] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 4415.628515] LNet: Accept secure, port 988 [ 4417.399393] Key type lgssc registered [ 4419.414475] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4465.673181] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4469.962199] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4486.548201] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4492.818852] Lustre: Mounted lustre-client [ 4501.608051] Lustre: lustre: root_squash is set to 0:0 [ 4501.646123] Lustre: lustre: nosquash_nids is cleared [ 4505.567937] WARNING: CPU: 3 PID: 62522 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 4505.571489] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 4505.587634] CPU: 3 PID: 62522 Comm: kworker/u8:2 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 4505.591568] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 4505.594513] Workqueue: writeback wb_workfn (flush-lustre-ffff9669fa) [ 4505.596776] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 4505.598796] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 4505.605813] RSP: 0018:ffffa71ec8febb18 EFLAGS: 00010202 [ 4505.607630] RAX: 0000000000000000 RBX: ffffa71ec8febc00 RCX: 0000000000000001 [ 4505.610165] RDX: 000000000000000e RSI: ffff9669eb0440c8 RDI: 0000000000000202 [ 4505.612625] RBP: ffff9669eb764290 R08: 0000000000000000 R09: 0000000000000001 [ 4505.615069] R10: 0000000003b98b82 R11: ffff966a00682000 R12: ffff9669eb7644d0 [ 4505.617770] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000 [ 4505.620241] FS: 0000000000000000(0000) GS:ffff966a01e00000(0000) knlGS:0000000000000000 [ 4505.623375] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4505.625417] CR2: 00007ff2e905a9c8 CR3: 0000000024228006 CR4: 0000000000170ee0 [ 4505.628243] Call Trace: [ 4505.629146] ? show_regs.cold.9+0x22/0x2f [ 4505.630542] ? __warn+0xc8/0x150 [ 4505.631708] ? ll_writepages+0x3dd/0x400 [lustre] [ 4505.633477] ? report_bug+0x113/0x140 [ 4505.634626] ? do_error_trap+0xb6/0x130 [ 4505.635724] ? do_invalid_op+0x46/0x60 [ 4505.637036] ? ll_writepages+0x3dd/0x400 [lustre] [ 4505.638776] ? invalid_op+0x14/0x20 [ 4505.640018] ? ll_writepages+0x3dd/0x400 [lustre] [ 4505.641840] ? ll_writepages+0x3b8/0x400 [lustre] [ 4505.643399] do_writepages+0x7a/0x270 [ 4505.644788] __writeback_single_inode+0xb2/0x7d0 [ 4505.646408] ? do_raw_spin_unlock+0x75/0x190 [ 4505.648000] writeback_sb_inodes+0x239/0x7e0 [ 4505.649568] __writeback_inodes_wb+0x71/0x110 [ 4505.651119] wb_writeback+0x627/0x7a0 [ 4505.652475] wb_workfn+0x709/0x950 [ 4505.653783] ? sched_clock_cpu+0x24/0x150 [ 4505.655205] process_one_work+0x3dd/0x9b0 [ 4505.656527] worker_thread+0x296/0x6e0 [ 4505.657648] ? rescuer_thread+0x570/0x570 [ 4505.658758] kthread+0x1d7/0x210 [ 4505.659795] ? set_kthread_struct+0x70/0x70 [ 4505.661174] ret_from_fork+0x1f/0x30 [ 4505.662450] ---[ end trace 4a77805e2a5b90b6 ]--- [ 4507.229902] Lustre: lustre: root_squash is set to 500:500 [ 4521.502093] Lustre: lustre: nosquash_nids set to 2@gni 0@lo 192.168.206.17@tcp 192.168.0.[2,10]@tcp [ 4530.210995] Lustre: lustre: nosquash_nids set to 192.168.206.17/24@tcp [ 4532.316314] Lustre: Unmounted lustre-client [ 4559.817101] Key type lgssc unregistered [ 4560.261579] LNet: 83666:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4561.321281] LNet: Removed LNI 192.168.206.17@tcp [ 4562.571689] Key type .llcrypt unregistered [ 4562.573478] Key type ._llcrypt unregistered [ 4582.257467] Key type ._llcrypt registered [ 4582.260582] Key type .llcrypt registered [ 4583.939533] Lustre: DEBUG MARKER: == conf-sanity test 43b: parse nosquash_nids with commas in expr_list ========================================================== 05:54:07 (1743501247) [ 4586.507810] Lustre: DEBUG MARKER: SKIP: conf-sanity test_43b mixed loopback and real device not working [ 4588.525079] Lustre: DEBUG MARKER: == conf-sanity test 44: mounted client proc entry exists ========================================================== 05:54:12 (1743501252) [ 4591.688526] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4591.709873] alg: No test for adler32 (adler32-zlib) [ 4592.942670] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 4593.270119] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 4593.272579] LNet: Accept secure, port 988 [ 4595.024884] Key type lgssc registered [ 4596.878146] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4639.291034] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4643.383616] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4659.653267] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4660.416716] Lustre: lustre: root_squash is set to 500:500 [ 4660.462323] Lustre: lustre: nosquash_nids set to 192.168.206.17/24@tcp [ 4665.884283] Lustre: Mounted lustre-client [ 4671.224846] Lustre: Unmounted lustre-client [ 4697.749552] Key type lgssc unregistered [ 4698.196612] LNet: 87378:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4699.241505] LNet: Removed LNI 192.168.206.17@tcp [ 4700.434939] Key type .llcrypt unregistered [ 4700.436635] Key type ._llcrypt unregistered [ 4718.933429] Key type ._llcrypt registered [ 4718.935071] Key type .llcrypt registered [ 4720.975341] Lustre: DEBUG MARKER: SKIP: conf-sanity test_45 skipping SLOW test 45 [ 4723.433230] Lustre: DEBUG MARKER: == conf-sanity test 46a: handle ost additional - wide striped file ========================================================== 05:56:26 (1743501386) [ 4740.446145] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4740.471288] alg: No test for adler32 (adler32-zlib) [ 4741.796440] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 4742.203070] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 4742.208471] LNet: Accept secure, port 988 [ 4743.975656] Key type lgssc registered [ 4746.042260] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4818.582147] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4822.859489] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4838.986527] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4901.443771] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4905.328734] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4917.671177] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4924.793111] Lustre: Mounted lustre-client [ 4944.498562] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff966a00729000.ost_server_uuid 50 [ 4946.190361] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff966a00729000.ost_server_uuid in FULL state after 0 sec [ 4946.669602] Lustre: Mounted lustre-client [ 4948.718903] WARNING: CPU: 1 PID: 12434 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 4948.722183] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 4948.732111] CPU: 1 PID: 12434 Comm: kworker/u8:1 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 4948.734340] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 4948.735989] Workqueue: writeback wb_workfn (flush-lustre-ffff9669f0) [ 4948.737219] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 4948.739007] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 4948.744464] RSP: 0018:ffffa71ec2023b58 EFLAGS: 00010202 [ 4948.745358] RAX: 0000000000000000 RBX: ffffa71ec2023c40 RCX: 0000000000000001 [ 4948.746987] RDX: 0000000000000007 RSI: ffff9669f96b10c8 RDI: 0000000000000202 [ 4948.748978] RBP: ffff9669ecc63250 R08: 0000000000000000 R09: 0000000000000001 [ 4948.750939] R10: 0000000004161bcb R11: ffff9669ed0d8000 R12: ffff9669ecc63490 [ 4948.752421] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 4948.753917] FS: 0000000000000000(0000) GS:ffff966a01600000(0000) knlGS:0000000000000000 [ 4948.755896] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4948.757241] CR2: 000056229f1c0a58 CR3: 0000000024228004 CR4: 0000000000170ee0 [ 4948.759506] Call Trace: [ 4948.760364] ? show_regs.cold.9+0x22/0x2f [ 4948.761804] ? __warn+0xc8/0x150 [ 4948.762551] ? ll_writepages+0x3dd/0x400 [lustre] [ 4948.764070] ? report_bug+0x113/0x140 [ 4948.764928] ? do_error_trap+0xb6/0x130 [ 4948.765651] ? do_invalid_op+0x46/0x60 [ 4948.766401] ? ll_writepages+0x3dd/0x400 [lustre] [ 4948.767865] ? invalid_op+0x14/0x20 [ 4948.768534] ? ll_writepages+0x3dd/0x400 [lustre] [ 4948.769439] ? ll_writepages+0x3b8/0x400 [lustre] [ 4948.770546] do_writepages+0x7a/0x270 [ 4948.771375] __writeback_single_inode+0xb2/0x7d0 [ 4948.772250] ? do_raw_spin_unlock+0x75/0x190 [ 4948.773087] writeback_sb_inodes+0x239/0x7e0 [ 4948.774241] wb_writeback+0x1ae/0x7a0 [ 4948.775524] wb_workfn+0x1e5/0x950 [ 4948.776781] ? lock_acquire+0x146/0x3a0 [ 4948.777623] ? process_one_work+0x2c0/0x9b0 [ 4948.778482] process_one_work+0x3dd/0x9b0 [ 4948.779248] worker_thread+0x296/0x6e0 [ 4948.780226] ? rescuer_thread+0x570/0x570 [ 4948.781527] kthread+0x1d7/0x210 [ 4948.782648] ? set_kthread_struct+0x70/0x70 [ 4948.783870] ret_from_fork+0x1f/0x30 [ 4948.784940] ---[ end trace 4a77805e2a5b90b7 ]--- [ 4948.874460] Lustre: Unmounted lustre-client [ 4949.992264] Lustre: Unmounted lustre-client [ 4987.863515] Key type lgssc unregistered [ 4988.293822] LNet: 94628:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4989.351249] LNet: Removed LNI 192.168.206.17@tcp [ 4990.569698] Key type .llcrypt unregistered [ 4990.570685] Key type ._llcrypt unregistered [ 5024.186838] Key type ._llcrypt registered [ 5024.188035] Key type .llcrypt registered [ 5024.511435] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 5024.533598] alg: No test for adler32 (adler32-zlib) [ 5025.709624] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 5026.029291] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 5026.031888] LNet: Accept secure, port 988 [ 5027.759338] Key type lgssc registered [ 5029.426655] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5088.190850] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5091.573371] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5105.018787] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5137.468557] Lustre: DEBUG MARKER: == conf-sanity test 47a: server restart does not lose lru_resize ========================================================== 06:03:21 (1743501801) [ 5202.401831] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5205.742352] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5220.262299] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5221.886121] Lustre: Mounted lustre-client [ 5242.341874] Lustre: lustre-OST0000-osc-ffff9669ed321000: Connection to lustre-OST0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 5243.359180] Lustre: 95425:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743501892/real 1743501892] req@000000003a6b6451 x1828194029147264/t0(0) o400->lustre-OST0000-osc-ffff9669ed321000@192.168.206.117@tcp:28/4 lens 224/224 e 0 to 1 dl 1743501908 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 5244.752782] Lustre: lustre-OST0000-osc-ffff9669ed321000: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 5247.461570] Lustre: lustre-MDT0000-mdc-ffff9669ed321000: Connection to lustre-MDT0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 5248.479164] Lustre: 95424:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743501897/real 1743501897] req@000000000e62274d x1828194029147776/t0(0) o400->lustre-OST0000-osc-ffff9669ed321000@192.168.206.117@tcp:28/4 lens 224/224 e 0 to 1 dl 1743501913 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 5253.599180] Lustre: 95424:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743501902/real 1743501902] req@0000000021206f6e x1828194029148288/t0(0) o400->lustre-OST0000-osc-ffff9669ed321000@192.168.206.117@tcp:28/4 lens 224/224 e 0 to 1 dl 1743501918 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 5262.815293] Lustre: 95424:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743501912/real 1743501912] req@000000003e685f7e x1828194029149312/t0(0) o400->MGC192.168.206.117@tcp@192.168.206.117@tcp:26/25 lens 224/224 e 0 to 1 dl 1743501928 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 5262.824665] LustreError: MGC192.168.206.117@tcp: Connection to MGS (at 192.168.206.117@tcp) was lost; in progress operations using this service will fail [ 5262.830394] Lustre: 102227:0:(mgc_request.c:1858:mgc_process_log()) MGC192.168.206.117@tcp: IR log lustre-cliir failed, not fatal: rc = -5 [ 5262.832756] Lustre: Evicted from MGS (at 192.168.206.117@tcp) after server handle changed from 0x2003433f3e1f1380 to 0x2003433f3e1f14f3 [ 5262.840831] Lustre: MGC192.168.206.117@tcp: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 5277.014244] Lustre: lustre-MDT0000-mdc-ffff9669ed321000: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 5278.464249] Lustre: Unmounted lustre-client [ 5302.390238] Key type lgssc unregistered [ 5302.783793] LNet: 103331:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5303.848486] LNet: Removed LNI 192.168.206.17@tcp [ 5304.942709] Key type .llcrypt unregistered [ 5304.944297] Key type ._llcrypt unregistered [ 5321.804047] Key type ._llcrypt registered [ 5321.805754] Key type .llcrypt registered [ 5323.248438] Lustre: DEBUG MARKER: == conf-sanity test 47b: client restart does not lose lru_resize ========================================================== 06:06:26 (1743501986) [ 5338.298194] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5338.319877] alg: No test for adler32 (adler32-zlib) [ 5339.542625] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 5339.854827] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 5339.857386] LNet: Accept secure, port 988 [ 5341.583335] Key type lgssc registered [ 5343.219976] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5372.728407] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5440.522830] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5444.063593] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5458.076633] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5459.720720] Lustre: Mounted lustre-client [ 5461.835967] Lustre: DEBUG MARKER: SKIP: conf-sanity test_47b Can not create symlink /usr/sbin/lctl [ 5463.603440] Lustre: DEBUG MARKER: == conf-sanity test 48: too many acls on file ============ 06:08:47 (1743502127) [ 5478.367564] WARNING: CPU: 3 PID: 38 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 5478.371545] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 5478.387554] CPU: 3 PID: 38 Comm: kworker/u8:3 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 5478.390877] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 5478.395177] Workqueue: writeback wb_workfn (flush-lustre-ffff9669f9) [ 5478.396985] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 5478.398208] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 5478.402913] RSP: 0018:ffffa71ec0dabb18 EFLAGS: 00010202 [ 5478.404554] RAX: 0000000000000000 RBX: ffffa71ec0dabc00 RCX: 0000000000000001 [ 5478.406700] RDX: 000000000000000e RSI: ffff9669f9da40c8 RDI: 0000000000000202 [ 5478.409191] RBP: ffff9669eb7d4290 R08: ffff9669eb7d4290 R09: 0000000000000001 [ 5478.412024] R10: ffffffffa0b77e50 R11: 0000000000000000 R12: ffff9669eb7d44d0 [ 5478.414768] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000 [ 5478.416962] FS: 0000000000000000(0000) GS:ffff966a01e00000(0000) knlGS:0000000000000000 [ 5478.419206] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5478.421170] CR2: 00007ff2fe7307e8 CR3: 0000000024228006 CR4: 0000000000170ee0 [ 5478.423328] Call Trace: [ 5478.424180] ? show_regs.cold.9+0x22/0x2f [ 5478.425563] ? __warn+0xc8/0x150 [ 5478.426699] ? ll_writepages+0x3dd/0x400 [lustre] [ 5478.428468] ? report_bug+0x113/0x140 [ 5478.429790] ? sched_clock_cpu+0x24/0x150 [ 5478.431230] ? do_error_trap+0xb6/0x130 [ 5478.432725] ? do_invalid_op+0x46/0x60 [ 5478.434070] ? ll_writepages+0x3dd/0x400 [lustre] [ 5478.435861] ? invalid_op+0x14/0x20 [ 5478.437143] ? writeback_sb_inodes+0x140/0x7e0 [ 5478.438806] ? ll_writepages+0x3dd/0x400 [lustre] [ 5478.440645] ? ll_writepages+0x3b8/0x400 [lustre] [ 5478.442276] do_writepages+0x7a/0x270 [ 5478.443611] __writeback_single_inode+0xb2/0x7d0 [ 5478.444540] ? do_raw_spin_unlock+0x75/0x190 [ 5478.445373] writeback_sb_inodes+0x239/0x7e0 [ 5478.446154] __writeback_inodes_wb+0x71/0x110 [ 5478.446980] wb_writeback+0x627/0x7a0 [ 5478.447668] wb_workfn+0x709/0x950 [ 5478.448316] ? sched_clock_cpu+0x24/0x150 [ 5478.449130] process_one_work+0x3dd/0x9b0 [ 5478.450080] worker_thread+0x296/0x6e0 [ 5478.451510] ? rescuer_thread+0x570/0x570 [ 5478.452885] kthread+0x1d7/0x210 [ 5478.453980] ? set_kthread_struct+0x70/0x70 [ 5478.455380] ret_from_fork+0x1f/0x30 [ 5478.456628] ---[ end trace 4a77805e2a5b90b8 ]--- [ 5596.671210] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 5607.002200] Lustre: Unmounted lustre-client [ 5695.413396] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5698.957796] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5713.615908] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5754.852269] Lustre: DEBUG MARKER: == conf-sanity test 49a: check PARAM_SYS_LDLM_TIMEOUT option of mkfs.lustre ========================================================== 06:13:38 (1743502418) [ 5823.942234] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5827.537371] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5841.919778] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5842.514888] Lustre: Mounted lustre-client [ 5845.179070] Lustre: Unmounted lustre-client [ 5871.822949] Lustre: DEBUG MARKER: == conf-sanity test 49b: check PARAM_SYS_LDLM_TIMEOUT option of mkfs.lustre ========================================================== 06:15:35 (1743502535) [ 5938.518981] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5942.183452] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5956.441381] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5957.087626] Lustre: Mounted lustre-client [ 5959.718423] Lustre: Unmounted lustre-client [ 5979.530679] Key type lgssc unregistered [ 5980.016140] LNet: 125110:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5981.030252] LNet: Removed LNI 192.168.206.17@tcp [ 5982.079624] Key type .llcrypt unregistered [ 5982.080570] Key type ._llcrypt unregistered [ 5998.899673] Key type ._llcrypt registered [ 5998.900898] Key type .llcrypt registered [ 6000.335449] Lustre: DEBUG MARKER: == conf-sanity test 50a: lazystatfs all servers available ========================================================== 06:17:44 (1743502664) [ 6003.523411] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6003.546690] alg: No test for adler32 (adler32-zlib) [ 6004.697343] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 6005.007346] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 6005.009926] LNet: Accept secure, port 988 [ 6006.759292] Key type lgssc registered [ 6008.311220] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6040.927112] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6044.122191] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6055.876874] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6057.545279] Lustre: Mounted lustre-client [ 6072.629812] WARNING: CPU: 1 PID: 110103 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 6072.633485] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 6072.649013] CPU: 1 PID: 110103 Comm: kworker/u8:2 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 6072.652672] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 6072.655161] Workqueue: writeback wb_workfn (flush-lustre-ffff9669f0) [ 6072.657426] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 6072.659282] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 6072.663697] RSP: 0018:ffffa71eca487b58 EFLAGS: 00010202 [ 6072.664905] RAX: 0000000000000000 RBX: ffffa71eca487c40 RCX: 0000000000000001 [ 6072.666516] RDX: 0000000000000007 RSI: ffff9669f98310c8 RDI: 0000000000000202 [ 6072.668057] RBP: ffff9669ecc82210 R08: 0000000000000000 R09: 0000000000000001 [ 6072.670295] R10: 0000000004726f54 R11: ffff9669eaa8c000 R12: ffff9669ecc82450 [ 6072.672310] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 6072.673789] FS: 0000000000000000(0000) GS:ffff966a01600000(0000) knlGS:0000000000000000 [ 6072.676382] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 6072.678162] CR2: 0000558f00c1df44 CR3: 0000000024228001 CR4: 0000000000170ee0 [ 6072.680219] Call Trace: [ 6072.680938] ? show_regs.cold.9+0x22/0x2f [ 6072.682094] ? __warn+0xc8/0x150 [ 6072.683063] ? ll_writepages+0x3dd/0x400 [lustre] [ 6072.684614] ? report_bug+0x113/0x140 [ 6072.685794] ? do_error_trap+0xb6/0x130 [ 6072.687063] ? do_invalid_op+0x46/0x60 [ 6072.688317] ? ll_writepages+0x3dd/0x400 [lustre] [ 6072.689946] ? invalid_op+0x14/0x20 [ 6072.691083] ? ll_writepages+0x3dd/0x400 [lustre] [ 6072.692130] ? ll_writepages+0x3b8/0x400 [lustre] [ 6072.693128] do_writepages+0x7a/0x270 [ 6072.693868] __writeback_single_inode+0xb2/0x7d0 [ 6072.694810] ? do_raw_spin_unlock+0x75/0x190 [ 6072.695690] writeback_sb_inodes+0x239/0x7e0 [ 6072.696553] wb_writeback+0x1ae/0x7a0 [ 6072.697276] wb_workfn+0x1e5/0x950 [ 6072.698075] ? lock_acquire+0x146/0x3a0 [ 6072.698855] ? process_one_work+0x2c0/0x9b0 [ 6072.700060] process_one_work+0x3dd/0x9b0 [ 6072.701115] worker_thread+0x296/0x6e0 [ 6072.702058] ? rescuer_thread+0x570/0x570 [ 6072.703067] kthread+0x1d7/0x210 [ 6072.703886] ? set_kthread_struct+0x70/0x70 [ 6072.704722] ret_from_fork+0x1f/0x30 [ 6072.705362] ---[ end trace 4a77805e2a5b90b9 ]--- [ 6072.780167] Lustre: Unmounted lustre-client [ 6105.547555] Key type lgssc unregistered [ 6106.057917] LNet: 128457:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6107.110931] LNet: Removed LNI 192.168.206.17@tcp [ 6108.178270] Key type .llcrypt unregistered [ 6108.179783] Key type ._llcrypt unregistered [ 6124.723195] Key type ._llcrypt registered [ 6124.724676] Key type .llcrypt registered [ 6126.108078] Lustre: DEBUG MARKER: == conf-sanity test 50b: lazystatfs all servers down ===== 06:19:49 (1743502789) [ 6129.255374] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6129.273709] alg: No test for adler32 (adler32-zlib) [ 6130.433623] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 6130.723416] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 6130.725176] LNet: Accept secure, port 988 [ 6132.463342] Key type lgssc registered [ 6133.991385] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6166.843640] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6170.091094] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6182.062329] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6183.626786] Lustre: Mounted lustre-client [ 6196.373737] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state DISCONN osc.lustre-OST0000-osc-ffff9669f0ea1000.ost_server_uuid 50 [ 6210.527256] Lustre: 129404:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743502859/real 1743502859] req@00000000dd7c4dd0 x1828195187824512/t0(0) o400->lustre-OST0000-osc-ffff9669f0ea1000@192.168.206.117@tcp:28/4 lens 224/224 e 0 to 1 dl 1743502875 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 6210.536283] Lustre: lustre-OST0000-osc-ffff9669f0ea1000: Connection to lustre-OST0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 6212.266238] Lustre: DEBUG MARKER: OSCs should all be DISCONN [ 6215.647242] Lustre: 129404:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743502864/real 1743502864] req@000000001f409857 x1828195187825024/t0(0) o400->lustre-OST0000-osc-ffff9669f0ea1000@192.168.206.117@tcp:28/4 lens 224/224 e 0 to 1 dl 1743502880 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 6219.743292] Lustre: 129404:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743502869/real 1743502869] req@00000000c6fb1da9 x1828195187825536/t0(0) o400->lustre-OST0000-osc-ffff9669f0ea1000@192.168.206.117@tcp:28/4 lens 224/224 e 0 to 1 dl 1743502885 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 6220.255434] WARNING: CPU: 2 PID: 110103 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 6220.258329] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 6220.273070] CPU: 2 PID: 110103 Comm: kworker/u8:2 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 6220.277512] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 6220.280223] Workqueue: writeback wb_workfn (flush-lustre-ffff9669f0) [ 6220.283665] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 6220.285499] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 6220.290788] RSP: 0018:ffffa71eca487b18 EFLAGS: 00010202 [ 6220.292466] RAX: 0000000000000000 RBX: ffffa71eca487c00 RCX: 0000000000000001 [ 6220.294837] RDX: 000000000000000e RSI: ffff9669f9df40c8 RDI: 0000000000000206 [ 6220.297249] RBP: ffff9669ecce91d0 R08: 0000000000000000 R09: 0000000000000001 [ 6220.299910] R10: 000000000483b4c8 R11: ffff966a00419000 R12: ffff9669ecce9410 [ 6220.302590] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000 [ 6220.305234] FS: 0000000000000000(0000) GS:ffff966a01a00000(0000) knlGS:0000000000000000 [ 6220.307675] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 6220.309223] CR2: 0000558f01991848 CR3: 0000000024228004 CR4: 0000000000170ee0 [ 6220.311378] Call Trace: [ 6220.311899] ? show_regs.cold.9+0x22/0x2f [ 6220.312736] ? __warn+0xc8/0x150 [ 6220.313630] ? ll_writepages+0x3dd/0x400 [lustre] [ 6220.315123] ? report_bug+0x113/0x140 [ 6220.316267] ? do_error_trap+0xb6/0x130 [ 6220.317487] ? do_invalid_op+0x46/0x60 [ 6220.318407] ? ll_writepages+0x3dd/0x400 [lustre] [ 6220.319919] ? invalid_op+0x14/0x20 [ 6220.320956] ? ll_writepages+0x3dd/0x400 [lustre] [ 6220.322664] ? ll_writepages+0x3b8/0x400 [lustre] [ 6220.324084] do_writepages+0x7a/0x270 [ 6220.325386] __writeback_single_inode+0xb2/0x7d0 [ 6220.326757] ? do_raw_spin_unlock+0x75/0x190 [ 6220.327908] writeback_sb_inodes+0x239/0x7e0 [ 6220.328961] __writeback_inodes_wb+0x71/0x110 [ 6220.329890] wb_writeback+0x627/0x7a0 [ 6220.331181] ? get_nr_inodes+0x45/0x70 [ 6220.332605] wb_workfn+0x778/0x950 [ 6220.333761] ? sched_clock_cpu+0x24/0x150 [ 6220.335192] process_one_work+0x3dd/0x9b0 [ 6220.336618] worker_thread+0x296/0x6e0 [ 6220.337838] ? rescuer_thread+0x570/0x570 [ 6220.339155] kthread+0x1d7/0x210 [ 6220.340220] ? set_kthread_struct+0x70/0x70 [ 6220.341649] ret_from_fork+0x1f/0x30 [ 6220.342863] ---[ end trace 4a77805e2a5b90ba ]--- [ 6222.564117] Lustre: Unmounted lustre-client [ 6241.372314] Lustre: DEBUG MARKER: == conf-sanity test 50c: lazystatfs one server down ====== 06:21:45 (1743502905) [ 6259.280132] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6262.169402] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6275.271675] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6288.444787] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 6289.142359] Lustre: Mounted lustre-client [ 6294.504482] Lustre: lustre-OST0000-osc-ffff9669f0b84000: Connection to lustre-OST0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 6312.693828] Lustre: Unmounted lustre-client [ 6349.372736] Lustre: DEBUG MARKER: == conf-sanity test 50d: lazystatfs client/server conn race ========================================================== 06:23:33 (1743503013) [ 6368.478303] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6371.857414] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6384.029351] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6396.566663] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 6397.303469] Lustre: Mounted lustre-client [ 6402.533685] Lustre: lustre-OST0000-osc-ffff9669f9574000: Connection to lustre-OST0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 6424.552528] Lustre: Unmounted lustre-client [ 6461.881913] Lustre: DEBUG MARKER: == conf-sanity test 50e: normal statfs all servers down == 06:25:25 (1743503125) [ 6529.748634] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6533.170421] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6546.692594] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6589.760638] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6592.873884] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6604.041674] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6622.579861] Lustre: Mounted lustre-client [ 6656.013764] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6668.766270] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 10 sec [ 6669.870525] Lustre: Unmounted lustre-client [ 6695.430620] Lustre: DEBUG MARKER: == conf-sanity test 50f: normal statfs one server in down ========================================================== 06:29:19 (1743503359) [ 6713.419775] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6716.986549] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6729.007336] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6747.380414] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 6766.464300] Lustre: Mounted lustre-client [ 6799.927718] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 6801.760976] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6807.524614] Lustre: lustre-OST0001-osc-ffff9669f170c000: Connection to lustre-OST0001 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 6811.065455] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 6821.399087] Lustre: Unmounted lustre-client [ 6864.277294] Lustre: DEBUG MARKER: == conf-sanity test 50g: deactivated OST should not cause panic ========================================================== 06:32:07 (1743503527) [ 6882.224540] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6885.613464] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6897.835988] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6898.596573] Lustre: Mounted lustre-client [ 6912.476266] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 6915.308188] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 1 sec [ 6923.230735] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9669f9829000.ost_server_uuid 50 [ 6924.903546] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9669f9829000.ost_server_uuid in FULL state after 0 sec [ 6926.545214] Lustre: Unmounted lustre-client [ 6926.863791] Lustre: setting import lustre-OST0001_UUID INACTIVE by administrator request [ 6926.865531] Lustre: Skipped 3 previous similar messages [ 6926.933045] Lustre: Mounted lustre-client [ 6927.739756] LustreError: 150626:0:(sec.c:404:import_sec_validate_get()) import 000000008847eb43 (NEW) with no sec [ 6967.915733] Lustre: DEBUG MARKER: == conf-sanity test 50h: LU-642: activate deactivated OST ========================================================== 06:33:51 (1743503631) [ 6986.533746] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6989.966727] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7002.265840] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7014.248399] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 7015.000843] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7015.147430] Lustre: Mounted lustre-client [ 7024.293162] Lustre: Unmounted lustre-client [ 7024.294815] Lustre: Skipped 1 previous similar message [ 7064.275423] Key type lgssc unregistered [ 7064.711276] LNet: 153861:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7065.770605] LNet: Removed LNI 192.168.206.17@tcp [ 7066.769371] Key type .llcrypt unregistered [ 7066.770517] Key type ._llcrypt unregistered [ 7082.718446] Key type ._llcrypt registered [ 7082.719404] Key type .llcrypt registered [ 7084.208839] Lustre: DEBUG MARKER: == conf-sanity test 50i: activate deactivated MDT ======== 06:35:47 (1743503747) [ 7086.915164] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7086.935830] alg: No test for adler32 (adler32-zlib) [ 7088.110731] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 7088.396684] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 7088.399386] LNet: Accept secure, port 988 [ 7090.119347] Key type lgssc registered [ 7091.690903] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7126.292687] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7129.569067] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7141.633248] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7153.519504] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 7154.002650] Lustre: setting import lustre-MDT0001_UUID INACTIVE by administrator request [ 7155.079807] Lustre: Mounted lustre-client [ 7160.305110] LustreError: lustre-MDT0001-mdc-ffff9669f0bec000: This client was evicted by lustre-MDT0001; in progress operations using this service will fail. [ 7160.311444] Lustre: lustre-MDT0001-mdc-ffff9669f0bec000: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 7163.500173] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7165.147797] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 7182.313774] Lustre: setting import lustre-MDT0001_UUID INACTIVE by administrator request [ 7184.687389] Lustre: Unmounted lustre-client [ 7212.966850] Lustre: DEBUG MARKER: == conf-sanity test 51: Verify that mdt_reint handles RMF_MDT_MD correctly when an OST is added ========================================================== 06:37:56 (1743503876) [ 7277.443847] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7280.470843] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7293.303313] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7293.945586] Lustre: Mounted lustre-client [ 7309.517778] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 7311.155917] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7317.477830] Lustre: lustre-OST0001-osc-ffff9669f920c000: Connection to lustre-OST0001 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 7324.099756] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [ 7334.376709] WARNING: CPU: 0 PID: 134498 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 7334.380132] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 7334.395072] CPU: 0 PID: 134498 Comm: kworker/u8:1 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 7334.397482] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 7334.399173] Workqueue: writeback wb_workfn (flush-lustre-ffff9669f9) [ 7334.400728] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 7334.401882] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 7334.405695] RSP: 0018:ffffa71ec8ebbb58 EFLAGS: 00010202 [ 7334.406686] RAX: 0000000000000000 RBX: ffffa71ec8ebbc40 RCX: 0000000000000001 [ 7334.407954] RDX: 0000000000000007 RSI: ffff9669fa2710c8 RDI: 0000000000000202 [ 7334.409476] RBP: ffff9669ecd211d0 R08: 0000000000000000 R09: 0000000000000001 [ 7334.410909] R10: 0000000005ef5d00 R11: ffff9669f0b2c000 R12: ffff9669ecd21410 [ 7334.412316] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 7334.413890] FS: 0000000000000000(0000) GS:ffff966a01200000(0000) knlGS:0000000000000000 [ 7334.415536] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 7334.416579] CR2: 00007f34353bfdf0 CR3: 0000000024228005 CR4: 0000000000170ef0 [ 7334.418110] Call Trace: [ 7334.418813] ? show_regs.cold.9+0x22/0x2f [ 7334.419555] ? __warn+0xc8/0x150 [ 7334.420581] ? ll_writepages+0x3dd/0x400 [lustre] [ 7334.421840] ? report_bug+0x113/0x140 [ 7334.423062] ? do_error_trap+0xb6/0x130 [ 7334.423830] ? do_invalid_op+0x46/0x60 [ 7334.424754] ? ll_writepages+0x3dd/0x400 [lustre] [ 7334.426104] ? invalid_op+0x14/0x20 [ 7334.426952] ? ll_writepages+0x3dd/0x400 [lustre] [ 7334.428106] ? ll_writepages+0x3b8/0x400 [lustre] [ 7334.429196] do_writepages+0x7a/0x270 [ 7334.430465] __writeback_single_inode+0xb2/0x7d0 [ 7334.431334] ? do_raw_spin_unlock+0x75/0x190 [ 7334.432684] writeback_sb_inodes+0x239/0x7e0 [ 7334.433646] wb_writeback+0x1ae/0x7a0 [ 7334.434965] wb_workfn+0x1e5/0x950 [ 7334.435627] ? lock_acquire+0x146/0x3a0 [ 7334.436749] ? process_one_work+0x2c0/0x9b0 [ 7334.437660] process_one_work+0x3dd/0x9b0 [ 7334.438554] worker_thread+0x296/0x6e0 [ 7334.439325] ? rescuer_thread+0x570/0x570 [ 7334.440544] kthread+0x1d7/0x210 [ 7334.441229] ? set_kthread_struct+0x70/0x70 [ 7334.442454] ret_from_fork+0x1f/0x30 [ 7334.443373] ---[ end trace 4a77805e2a5b90bb ]--- [ 7334.503412] Lustre: Unmounted lustre-client [ 7372.695776] Key type lgssc unregistered [ 7373.164209] LNet: 162887:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7374.182965] LNet: Removed LNI 192.168.206.17@tcp [ 7375.171185] Key type .llcrypt unregistered [ 7375.173078] Key type ._llcrypt unregistered [ 7405.803467] Key type ._llcrypt registered [ 7405.805057] Key type .llcrypt registered [ 7406.106878] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 7406.126686] alg: No test for adler32 (adler32-zlib) [ 7407.266776] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 7407.538864] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 7407.541680] LNet: Accept secure, port 988 [ 7409.247322] Key type lgssc registered [ 7411.157488] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7472.376847] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7475.806496] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7490.172362] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7523.984649] Lustre: DEBUG MARKER: == conf-sanity test 52: check recovering objects from lost+found ========================================================== 06:43:07 (1743504187) [ 7542.994374] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7546.459511] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7558.997076] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7560.695705] Lustre: Mounted lustre-client [ 7604.407430] WARNING: CPU: 0 PID: 147123 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 7604.411364] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 7604.421616] CPU: 0 PID: 147123 Comm: kworker/u8:0 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 7604.424578] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 7604.426440] Workqueue: writeback wb_workfn (flush-lustre-ffff966a00) [ 7604.429818] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 7604.431574] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 7604.439052] RSP: 0018:ffffa71ec8967b58 EFLAGS: 00010202 [ 7604.440756] RAX: 0000000000000000 RBX: ffffa71ec8967c40 RCX: 0000000000000001 [ 7604.442809] RDX: 0000000000000007 RSI: ffff966a00bf10c8 RDI: 0000000000000206 [ 7604.444961] RBP: ffff9669eccb4290 R08: 0000000000000000 R09: 0000000000000001 [ 7604.447141] R10: 0000000009218b53 R11: ffff9669f08ee000 R12: ffff9669eccb44d0 [ 7604.449241] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 7604.451280] FS: 0000000000000000(0000) GS:ffff966a01200000(0000) knlGS:0000000000000000 [ 7604.453558] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 7604.455135] CR2: 0000558bc0b88028 CR3: 0000000024228002 CR4: 0000000000170ef0 [ 7604.457177] Call Trace: [ 7604.457897] ? show_regs.cold.9+0x22/0x2f [ 7604.459207] ? __warn+0xc8/0x150 [ 7604.460146] ? ll_writepages+0x3dd/0x400 [lustre] [ 7604.461599] ? report_bug+0x113/0x140 [ 7604.462712] ? do_error_trap+0xb6/0x130 [ 7604.463917] ? do_invalid_op+0x46/0x60 [ 7604.465302] ? ll_writepages+0x3dd/0x400 [lustre] [ 7604.467770] ? invalid_op+0x14/0x20 [ 7604.469814] ? ll_writepages+0x3dd/0x400 [lustre] [ 7604.472595] ? ll_writepages+0x3b8/0x400 [lustre] [ 7604.475395] do_writepages+0x7a/0x270 [ 7604.477476] __writeback_single_inode+0xb2/0x7d0 [ 7604.480125] ? do_raw_spin_unlock+0x75/0x190 [ 7604.482540] writeback_sb_inodes+0x239/0x7e0 [ 7604.484619] wb_writeback+0x1ae/0x7a0 [ 7604.485454] wb_workfn+0x1e5/0x950 [ 7604.486130] ? lock_acquire+0x146/0x3a0 [ 7604.486871] ? process_one_work+0x2c0/0x9b0 [ 7604.487782] process_one_work+0x3dd/0x9b0 [ 7604.488634] worker_thread+0x296/0x6e0 [ 7604.489406] ? rescuer_thread+0x570/0x570 [ 7604.490173] kthread+0x1d7/0x210 [ 7604.490888] ? set_kthread_struct+0x70/0x70 [ 7604.491655] ret_from_fork+0x1f/0x30 [ 7604.492311] ---[ end trace 4a77805e2a5b90bc ]--- [ 7604.925161] Lustre: Unmounted lustre-client [ 7631.829444] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7632.650768] Lustre: Mounted lustre-client [ 7667.800469] Lustre: Unmounted lustre-client [ 7692.804716] Key type lgssc unregistered [ 7693.203440] LNet: 169816:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7694.247518] LNet: Removed LNI 192.168.206.17@tcp [ 7695.580320] Key type .llcrypt unregistered [ 7695.581709] Key type ._llcrypt unregistered [ 7712.860906] Key type ._llcrypt registered [ 7712.862176] Key type .llcrypt registered [ 7714.260715] Lustre: DEBUG MARKER: == conf-sanity test 53a: check OSS thread count params === 06:46:18 (1743504378) [ 7718.459247] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7718.487101] alg: No test for adler32 (adler32-zlib) [ 7719.840932] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 7720.240958] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 7720.242808] LNet: Accept secure, port 988 [ 7722.007357] Key type lgssc registered [ 7723.933558] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7756.187452] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7759.853421] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7771.468789] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7773.147435] Lustre: Mounted lustre-client [ 7787.786405] Lustre: Unmounted lustre-client [ 7807.725749] Key type lgssc unregistered [ 7808.225499] LNet: 173321:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7809.254601] LNet: Removed LNI 192.168.206.17@tcp [ 7810.528730] Key type .llcrypt unregistered [ 7810.530169] Key type ._llcrypt unregistered [ 7826.459725] Key type ._llcrypt registered [ 7826.460682] Key type .llcrypt registered [ 7826.837206] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 7826.857323] alg: No test for adler32 (adler32-zlib) [ 7828.135408] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 7828.498521] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 7828.501383] LNet: Accept secure, port 988 [ 7830.303465] Key type lgssc registered [ 7832.286870] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7866.098316] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7869.942496] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7881.854569] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7883.531557] Lustre: Mounted lustre-client [ 7891.887128] Lustre: Unmounted lustre-client [ 7915.751705] Key type lgssc unregistered [ 7916.252767] LNet: 176141:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7917.288288] LNet: Removed LNI 192.168.206.17@tcp [ 7918.398489] Key type .llcrypt unregistered [ 7918.399360] Key type ._llcrypt unregistered [ 7936.934359] Key type ._llcrypt registered [ 7936.935450] Key type .llcrypt registered [ 7938.450913] Lustre: DEBUG MARKER: == conf-sanity test 53b: check MDS thread count params === 06:50:02 (1743504602) [ 7943.439100] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7943.465285] alg: No test for adler32 (adler32-zlib) [ 7944.659401] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 7944.956479] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 7944.958494] LNet: Accept secure, port 988 [ 7946.703457] Key type lgssc registered [ 7948.453319] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7983.372428] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7987.504626] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8000.423095] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8002.205555] Lustre: Mounted lustre-client [ 8013.801183] Lustre: Unmounted lustre-client [ 8047.946839] Key type lgssc unregistered [ 8048.450420] LNet: 179650:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8049.510579] LNet: Removed LNI 192.168.206.17@tcp [ 8050.805284] Key type .llcrypt unregistered [ 8050.806167] Key type ._llcrypt unregistered [ 8065.956500] Key type ._llcrypt registered [ 8065.957673] Key type .llcrypt registered [ 8066.409379] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 8066.429712] alg: No test for adler32 (adler32-zlib) [ 8067.558871] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 8067.918868] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 8067.921452] LNet: Accept secure, port 988 [ 8069.631349] Key type lgssc registered [ 8071.777720] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8104.852482] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8108.531764] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8120.058804] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8121.936781] Lustre: Mounted lustre-client [ 8130.138339] Lustre: Unmounted lustre-client [ 8155.889450] Key type lgssc unregistered [ 8156.394805] LNet: 182469:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8157.414749] LNet: Removed LNI 192.168.206.17@tcp [ 8158.631920] Key type .llcrypt unregistered [ 8158.633505] Key type ._llcrypt unregistered [ 8173.927698] Key type ._llcrypt registered [ 8173.928601] Key type .llcrypt registered [ 8175.176715] Lustre: DEBUG MARKER: == conf-sanity test 54a: test llverdev and partial verify of device ========================================================== 06:53:59 (1743504839) [ 8189.950692] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8189.969855] alg: No test for adler32 (adler32-zlib) [ 8191.110590] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 8191.399215] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 8191.401748] LNet: Accept secure, port 988 [ 8193.111361] Key type lgssc registered [ 8194.744647] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8249.922645] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8253.235768] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8266.296767] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8306.076778] Lustre: DEBUG MARKER: == conf-sanity test 54b: test llverfs and partial verify of filesystem ========================================================== 06:56:09 (1743504969) [ 8324.089813] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8327.387388] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8339.028742] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8340.558532] Lustre: Mounted lustre-client [ 8346.881134] WARNING: CPU: 2 PID: 147123 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 8346.883481] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 8346.891318] CPU: 2 PID: 147123 Comm: kworker/u8:0 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 8346.893812] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 8346.896677] Workqueue: writeback wb_workfn (flush-lustre-ffff966a00) [ 8346.899910] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 8346.901800] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 8346.907623] RSP: 0018:ffffa71ec8967b58 EFLAGS: 00010202 [ 8346.908512] RAX: 0000000000000000 RBX: ffffa71ec8967c40 RCX: 0000000000000001 [ 8346.909648] RDX: 0000000000000007 RSI: ffff9669f11790c8 RDI: 0000000000000202 [ 8346.910797] RBP: ffff9669d8300190 R08: 0000000000000000 R09: 0000000000000001 [ 8346.912114] R10: 000000000b505f1c R11: ffff9669ea9c7000 R12: ffff9669d83003d0 [ 8346.913351] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 8346.914595] FS: 0000000000000000(0000) GS:ffff966a01a00000(0000) knlGS:0000000000000000 [ 8346.916046] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 8346.917142] CR2: 00007f07572842a0 CR3: 0000000024228003 CR4: 0000000000170ee0 [ 8346.918543] Call Trace: [ 8346.918998] ? show_regs.cold.9+0x22/0x2f [ 8346.919727] ? __warn+0xc8/0x150 [ 8346.920290] ? ll_writepages+0x3dd/0x400 [lustre] [ 8346.921211] ? report_bug+0x113/0x140 [ 8346.921951] ? do_error_trap+0xb6/0x130 [ 8346.922707] ? do_invalid_op+0x46/0x60 [ 8346.923411] ? ll_writepages+0x3dd/0x400 [lustre] [ 8346.924916] ? invalid_op+0x14/0x20 [ 8346.926122] ? ll_writepages+0x3dd/0x400 [lustre] [ 8346.927796] ? ll_writepages+0x3b8/0x400 [lustre] [ 8346.929508] do_writepages+0x7a/0x270 [ 8346.930748] __writeback_single_inode+0xb2/0x7d0 [ 8346.932318] ? do_raw_spin_unlock+0x75/0x190 [ 8346.933730] writeback_sb_inodes+0x239/0x7e0 [ 8346.935181] wb_writeback+0x1ae/0x7a0 [ 8346.936487] wb_workfn+0x1e5/0x950 [ 8346.937647] ? lock_acquire+0x146/0x3a0 [ 8346.938672] ? process_one_work+0x2c0/0x9b0 [ 8346.939410] process_one_work+0x3dd/0x9b0 [ 8346.940069] worker_thread+0x296/0x6e0 [ 8346.940759] ? rescuer_thread+0x570/0x570 [ 8346.941421] kthread+0x1d7/0x210 [ 8346.941945] ? set_kthread_struct+0x70/0x70 [ 8346.942662] ret_from_fork+0x1f/0x30 [ 8346.943280] ---[ end trace 4a77805e2a5b90bd ]--- [ 8347.146342] Lustre: Unmounted lustre-client [ 8366.831929] Key type lgssc unregistered [ 8367.240924] LNet: 189183:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8368.294772] LNet: Removed LNI 192.168.206.17@tcp [ 8369.331903] Key type .llcrypt unregistered [ 8369.332947] Key type ._llcrypt unregistered [ 8384.395237] Key type ._llcrypt registered [ 8384.396255] Key type .llcrypt registered [ 8386.083827] Lustre: DEBUG MARKER: SKIP: conf-sanity test_55 skipping excluded test 55 [ 8388.024932] Lustre: DEBUG MARKER: == conf-sanity test 56a: check big OST indexes and out-of-index-order start ========================================================== 06:57:31 (1743505051) [ 8402.044171] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8402.071922] alg: No test for adler32 (adler32-zlib) [ 8403.392822] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 8403.786375] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 8403.789497] LNet: Accept secure, port 988 [ 8405.551347] Key type lgssc registered [ 8407.380864] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8474.696594] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8478.533374] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8493.275028] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid [ 8506.611932] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid [ 8508.257091] Lustre: Mounted lustre-client [ 8514.015471] WARNING: CPU: 2 PID: 110103 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 8514.019010] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 8514.032875] CPU: 2 PID: 110103 Comm: kworker/u8:2 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 8514.036847] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 8514.038951] Workqueue: writeback wb_workfn (flush-lustre-ffff9669fb) [ 8514.040173] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 8514.041198] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 8514.044274] RSP: 0018:ffffa71eca487b18 EFLAGS: 00010202 [ 8514.045171] RAX: 0000000000000000 RBX: ffffa71eca487c00 RCX: 0000000000000001 [ 8514.046322] RDX: 000000000000000e RSI: ffff9669f0cec0c8 RDI: 0000000000000206 [ 8514.047596] RBP: ffff9669ecc38190 R08: 0000000000000000 R09: 0000000000000001 [ 8514.049235] R10: 000000000b74b817 R11: ffff9669f14ac000 R12: ffff9669ecc383d0 [ 8514.050378] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000 [ 8514.051564] FS: 0000000000000000(0000) GS:ffff966a01a00000(0000) knlGS:0000000000000000 [ 8514.052933] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 8514.053839] CR2: 0000555aeb7d32c8 CR3: 0000000024228004 CR4: 0000000000170ee0 [ 8514.055088] Call Trace: [ 8514.055646] ? show_regs.cold.9+0x22/0x2f [ 8514.056707] ? __warn+0xc8/0x150 [ 8514.057420] ? ll_writepages+0x3dd/0x400 [lustre] [ 8514.058950] ? report_bug+0x113/0x140 [ 8514.060104] ? do_error_trap+0xb6/0x130 [ 8514.061439] ? do_invalid_op+0x46/0x60 [ 8514.062479] ? ll_writepages+0x3dd/0x400 [lustre] [ 8514.063448] ? invalid_op+0x14/0x20 [ 8514.064105] ? ll_writepages+0x3dd/0x400 [lustre] [ 8514.064953] ? ll_writepages+0x3b8/0x400 [lustre] [ 8514.065839] do_writepages+0x7a/0x270 [ 8514.066478] __writeback_single_inode+0xb2/0x7d0 [ 8514.067460] ? do_raw_spin_unlock+0x75/0x190 [ 8514.069068] writeback_sb_inodes+0x239/0x7e0 [ 8514.070574] __writeback_inodes_wb+0x71/0x110 [ 8514.072032] wb_writeback+0x627/0x7a0 [ 8514.073425] wb_workfn+0x709/0x950 [ 8514.074696] ? sched_clock_cpu+0x24/0x150 [ 8514.076129] process_one_work+0x3dd/0x9b0 [ 8514.077402] worker_thread+0x296/0x6e0 [ 8514.078744] ? rescuer_thread+0x570/0x570 [ 8514.080121] kthread+0x1d7/0x210 [ 8514.081003] ? set_kthread_struct+0x70/0x70 [ 8514.081956] ret_from_fork+0x1f/0x30 [ 8514.082731] ---[ end trace 4a77805e2a5b90be ]--- [ 8520.536556] Lustre: Unmounted lustre-client [ 8626.050727] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8629.457523] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8642.213743] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8675.712588] Lustre: DEBUG MARKER: == conf-sanity test 56b: test target_obd correctness with nonconsecutive MDTs ========================================================== 07:02:19 (1743505339) [ 8677.930876] Lustre: DEBUG MARKER: SKIP: conf-sanity test_56b needs >= 3 MDTs [ 8680.292220] Lustre: DEBUG MARKER: == conf-sanity test 57a: initial registration from failnode should fail (should return errs) ========================================================== 07:02:23 (1743505343) [ 8713.289085] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8716.888179] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8739.630431] Key type lgssc unregistered [ 8740.051715] LNet: 200530:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8741.094239] LNet: Removed LNI 192.168.206.17@tcp [ 8742.257430] Key type .llcrypt unregistered [ 8742.258739] Key type ._llcrypt unregistered [ 8759.215846] Key type ._llcrypt registered [ 8759.216794] Key type .llcrypt registered [ 8760.515157] Lustre: DEBUG MARKER: == conf-sanity test 57b: initial registration from servicenode should not fail ========================================================== 07:03:44 (1743505424) [ 8783.490502] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8783.518539] alg: No test for adler32 (adler32-zlib) [ 8784.675591] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 8785.044547] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 8785.046358] LNet: Accept secure, port 988 [ 8786.799240] Key type lgssc registered [ 8788.399363] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8816.713816] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8820.175274] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8831.699181] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8855.598152] Key type lgssc unregistered [ 8855.980795] LNet: 204012:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8855.987740] LNet: Removed LNI 192.168.206.17@tcp [ 8857.011982] Key type .llcrypt unregistered [ 8857.012918] Key type ._llcrypt unregistered [ 8874.269442] Key type ._llcrypt registered [ 8874.270447] Key type .llcrypt registered [ 8876.002589] Lustre: DEBUG MARKER: == conf-sanity test 58: missing llog files must not prevent MDT from mounting ========================================================== 07:05:39 (1743505539) [ 8879.350511] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8879.370781] alg: No test for adler32 (adler32-zlib) [ 8880.598665] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 8881.006797] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 8881.008857] LNet: Accept secure, port 988 [ 8882.743340] Key type lgssc registered [ 8884.766100] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8920.674097] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8924.829911] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8937.247920] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8939.039072] Lustre: Mounted lustre-client [ 8944.099588] Lustre: lustre-MDT0000-mdc-ffff9669fbca4000: Connection to lustre-MDT0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 8949.219912] Lustre: lustre-MDT0001-mdc-ffff9669fbca4000: Connection to lustre-MDT0001 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 8964.578090] LustreError: MGC192.168.206.117@tcp: Connection to MGS (at 192.168.206.117@tcp) was lost; in progress operations using this service will fail [ 8964.584477] Lustre: Evicted from MGS (at 192.168.206.117@tcp) after server handle changed from 0x9ef122780ed9462 to 0x9ef122780edadc2 [ 8964.588647] Lustre: MGC192.168.206.117@tcp: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 8984.037140] LustreError: lustre-MDT0000-mdc-ffff9669fbca4000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. [ 8984.099693] Lustre: lustre-MDT0000-mdc-ffff9669fbca4000: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 8986.686914] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8988.760256] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 8992.123972] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8994.450063] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 8996.066198] Lustre: Unmounted lustre-client [ 9029.006106] Key type lgssc unregistered [ 9029.482178] LNet: 208648:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9030.502847] LNet: Removed LNI 192.168.206.17@tcp [ 9031.696610] Key type .llcrypt unregistered [ 9031.697674] Key type ._llcrypt unregistered [ 9050.146861] Key type ._llcrypt registered [ 9050.147936] Key type .llcrypt registered [ 9052.134913] Lustre: DEBUG MARKER: == conf-sanity test 59: writeconf mount option =========== 07:08:35 (1743505715) [ 9057.804076] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9057.837748] alg: No test for adler32 (adler32-zlib) [ 9059.074984] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 9059.457783] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 9059.459747] LNet: Accept secure, port 988 [ 9061.215435] Key type lgssc registered [ 9063.356646] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9100.331814] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9103.956311] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9139.622982] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9143.246592] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9160.337895] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9172.747904] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 9219.568478] Key type lgssc unregistered [ 9220.156432] LNet: 213557:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9220.164422] LNet: Removed LNI 192.168.206.17@tcp [ 9221.541278] Key type .llcrypt unregistered [ 9221.542725] Key type ._llcrypt unregistered [ 9258.334389] Key type ._llcrypt registered [ 9258.335867] Key type .llcrypt registered [ 9258.782325] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 9258.802929] alg: No test for adler32 (adler32-zlib) [ 9259.991102] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 9260.397179] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 9260.399791] LNet: Accept secure, port 988 [ 9262.119453] Key type lgssc registered [ 9263.950838] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9322.540546] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9326.361072] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9340.613261] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9373.643183] Lustre: DEBUG MARKER: == conf-sanity test 60a: check mkfs.lustre --mkfsoptions -E -O options setting ========================================================== 07:13:57 (1743506037) [ 9455.370413] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9458.778073] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9473.631130] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9506.770433] Lustre: DEBUG MARKER: == conf-sanity test 60b: check mkfs.lustre MDT default features ========================================================== 07:16:10 (1743506170) [ 9515.481451] Lustre: DEBUG MARKER: == conf-sanity test 61a: large xattr ===================== 07:16:19 (1743506179) [ 9537.095744] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9540.418829] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9552.991663] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9554.845649] Lustre: Mounted lustre-client [ 9560.411525] Lustre: DEBUG MARKER: save large xattr of 65536 bytes on trusted.big on /mnt/lustre/f61a.conf-sanity [ 9562.279254] Lustre: DEBUG MARKER: shrink value of trusted.big on /mnt/lustre/f61a.conf-sanity [ 9564.133657] Lustre: DEBUG MARKER: grow value of trusted.big on /mnt/lustre/f61a.conf-sanity [ 9565.866280] Lustre: DEBUG MARKER: check value of trusted.big on /mnt/lustre/f61a.conf-sanity after remounting MDS [ 9570.276503] Lustre: lustre-MDT0000-mdc-ffff9669f13f4000: Connection to lustre-MDT0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 9585.631189] Lustre: 214345:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1743506235/real 1743506235] req@00000000d2a74d49 x1828198468822784/t0(0) o400->MGC192.168.206.117@tcp@192.168.206.117@tcp:26/25 lens 224/224 e 0 to 1 dl 1743506251 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 9585.637635] LustreError: MGC192.168.206.117@tcp: Connection to MGS (at 192.168.206.117@tcp) was lost; in progress operations using this service will fail [ 9585.651113] Lustre: Evicted from MGS (at 192.168.206.117@tcp) after server handle changed from 0x40086f45f584fabf to 0x40086f45f584fd43 [ 9585.658804] Lustre: MGC192.168.206.117@tcp: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 9588.556361] Lustre: lustre-MDT0000-mdc-ffff9669f13f4000: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 9589.215364] WARNING: CPU: 1 PID: 147123 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 9589.217918] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 9589.229494] CPU: 1 PID: 147123 Comm: kworker/u8:0 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 9589.231777] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 9589.234579] Workqueue: writeback wb_workfn (flush-lustre-ffff9669f1) [ 9589.236814] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 9589.238103] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 9589.241906] RSP: 0018:ffffa71ec8967b18 EFLAGS: 00010202 [ 9589.243195] RAX: 0000000000000000 RBX: ffffa71ec8967c00 RCX: 0000000000000001 [ 9589.244861] RDX: 000000000000000e RSI: ffff9669f91540c8 RDI: 0000000000000202 [ 9589.246328] RBP: ffff9669ecc78190 R08: 0000000000000000 R09: 0000000000000001 [ 9589.247806] R10: 000000000c0a64d3 R11: ffff9669f1266000 R12: ffff9669ecc783d0 [ 9589.249192] R13: ffffffffffffffff R14: 0000000000000000 R15: 0000000000000000 [ 9589.250919] FS: 0000000000000000(0000) GS:ffff966a01600000(0000) knlGS:0000000000000000 [ 9589.252812] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 9589.253975] CR2: 00007ffd7b74c098 CR3: 0000000024228003 CR4: 0000000000170ee0 [ 9589.255375] Call Trace: [ 9589.256199] ? show_regs.cold.9+0x22/0x2f [ 9589.257161] ? __warn+0xc8/0x150 [ 9589.257911] ? ll_writepages+0x3dd/0x400 [lustre] [ 9589.259160] ? report_bug+0x113/0x140 [ 9589.259985] ? do_error_trap+0xb6/0x130 [ 9589.260842] ? do_invalid_op+0x46/0x60 [ 9589.261920] ? ll_writepages+0x3dd/0x400 [lustre] [ 9589.263107] ? invalid_op+0x14/0x20 [ 9589.264019] ? ll_writepages+0x3dd/0x400 [lustre] [ 9589.265134] ? ll_writepages+0x3b8/0x400 [lustre] [ 9589.266180] do_writepages+0x7a/0x270 [ 9589.267239] __writeback_single_inode+0xb2/0x7d0 [ 9589.268311] ? do_raw_spin_unlock+0x75/0x190 [ 9589.269594] writeback_sb_inodes+0x239/0x7e0 [ 9589.270725] __writeback_inodes_wb+0x71/0x110 [ 9589.271634] wb_writeback+0x627/0x7a0 [ 9589.272747] ? get_nr_inodes+0x45/0x70 [ 9589.273720] wb_workfn+0x778/0x950 [ 9589.274503] ? sched_clock_cpu+0x24/0x150 [ 9589.275621] process_one_work+0x3dd/0x9b0 [ 9589.276477] worker_thread+0x296/0x6e0 [ 9589.277182] ? rescuer_thread+0x570/0x570 [ 9589.278232] kthread+0x1d7/0x210 [ 9589.279050] ? set_kthread_struct+0x70/0x70 [ 9589.280167] ret_from_fork+0x1f/0x30 [ 9589.281222] ---[ end trace 4a77805e2a5b90bf ]--- [ 9591.811347] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9593.818989] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 9597.364677] Lustre: DEBUG MARKER: remove large xattr trusted.big from /mnt/lustre/f61a.conf-sanity [ 9600.995568] Lustre: lustre-MDT0000-mdc-ffff9669f13f4000: Connection to lustre-MDT0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [ 9616.358121] LustreError: MGC192.168.206.117@tcp: Connection to MGS (at 192.168.206.117@tcp) was lost; in progress operations using this service will fail [ 9616.371037] Lustre: Evicted from MGS (at 192.168.206.117@tcp) after server handle changed from 0x40086f45f584fd43 to 0x40086f45f5850045 [ 9616.376585] Lustre: MGC192.168.206.117@tcp: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 9620.299837] Lustre: lustre-MDT0000-mdc-ffff9669f13f4000: Connection restored to 192.168.206.117@tcp (at 192.168.206.117@tcp) [ 9623.896823] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9626.380499] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 9629.277950] Lustre: Unmounted lustre-client [ 9679.404685] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9683.250788] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9696.698434] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9712.638437] Lustre: Mounted lustre-client [ 9714.251288] Lustre: Unmounted lustre-client [ 9748.008586] Key type lgssc unregistered [ 9748.444170] LNet: 228785:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9749.479252] LNet: Removed LNI 192.168.206.17@tcp [ 9750.658727] Key type .llcrypt unregistered [ 9750.659794] Key type ._llcrypt unregistered [ 9769.578468] Key type ._llcrypt registered [ 9769.579610] Key type .llcrypt registered [ 9771.534853] Lustre: DEBUG MARKER: == conf-sanity test 61b: large xattr ===================== 07:20:34 (1743506434) [ 9776.083354] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9776.107142] alg: No test for adler32 (adler32-zlib) [ 9777.222887] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [ 9777.530840] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [ 9777.532725] LNet: Accept secure, port 988 [ 9779.327393] Key type lgssc registered [ 9780.962740] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9815.460835] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9819.033730] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9831.509252] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9833.125106] Lustre: Mounted lustre-client [ 9841.355968] WARNING: CPU: 0 PID: 218031 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [ 9841.358331] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [ 9841.368481] CPU: 0 PID: 218031 Comm: kworker/u8:3 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [ 9841.371015] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [ 9841.372858] Workqueue: writeback wb_workfn (flush-lustre-ffff9669f9) [ 9841.375286] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [ 9841.376514] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [ 9841.380476] RSP: 0018:ffffa71ec8b8bb58 EFLAGS: 00010202 [ 9841.381534] RAX: 0000000000000000 RBX: ffffa71ec8b8bc40 RCX: 0000000000000001 [ 9841.382819] RDX: 0000000000000007 RSI: ffff9669fca4c0c8 RDI: 0000000000000202 [ 9841.384170] RBP: ffff9669ecc7e310 R08: 0000000000000000 R09: 0000000000000001 [ 9841.385438] R10: 000000000c315d12 R11: ffff9669f9731000 R12: ffff9669ecc7e550 [ 9841.386792] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [ 9841.388261] FS: 0000000000000000(0000) GS:ffff966a01200000(0000) knlGS:0000000000000000 [ 9841.389960] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 9841.391046] CR2: 00005600b5c89028 CR3: 0000000024228006 CR4: 0000000000170ef0 [ 9841.392496] Call Trace: [ 9841.393005] ? show_regs.cold.9+0x22/0x2f [ 9841.393770] ? __warn+0xc8/0x150 [ 9841.394475] ? ll_writepages+0x3dd/0x400 [lustre] [ 9841.395500] ? report_bug+0x113/0x140 [ 9841.396243] ? do_error_trap+0xb6/0x130 [ 9841.397072] ? do_invalid_op+0x46/0x60 [ 9841.397796] ? ll_writepages+0x3dd/0x400 [lustre] [ 9841.398764] ? invalid_op+0x14/0x20 [ 9841.399536] ? ll_writepages+0x3dd/0x400 [lustre] [ 9841.400459] ? ll_writepages+0x3b8/0x400 [lustre] [ 9841.401507] do_writepages+0x7a/0x270 [ 9841.402214] __writeback_single_inode+0xb2/0x7d0 [ 9841.403102] ? do_raw_spin_unlock+0x75/0x190 [ 9841.403930] writeback_sb_inodes+0x239/0x7e0 [ 9841.404796] wb_writeback+0x1ae/0x7a0 [ 9841.405597] wb_workfn+0x1e5/0x950 [ 9841.406309] ? lock_acquire+0x146/0x3a0 [ 9841.407113] ? process_one_work+0x2c0/0x9b0 [ 9841.407942] process_one_work+0x3dd/0x9b0 [ 9841.408705] worker_thread+0x296/0x6e0 [ 9841.409450] ? rescuer_thread+0x570/0x570 [ 9841.410258] kthread+0x1d7/0x210 [ 9841.410893] ? set_kthread_struct+0x70/0x70 [ 9841.411708] ret_from_fork+0x1f/0x30 [ 9841.412420] ---[ end trace 4a77805e2a5b90c0 ]--- [ 9841.493830] Lustre: Unmounted lustre-client [ 9890.162386] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9894.586468] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9907.721832] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9908.725954] Lustre: Mounted lustre-client [ 9916.252074] Lustre: Unmounted lustre-client [ 9965.156565] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9969.520541] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9982.481421] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9983.438393] Lustre: Mounted lustre-client [ 9984.983389] Lustre: Unmounted lustre-client [10006.927708] Key type lgssc unregistered [10007.618710] LNet: 235339:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10008.680468] LNet: Removed LNI 192.168.206.17@tcp [10010.316344] Key type .llcrypt unregistered [10010.317486] Key type ._llcrypt unregistered [10032.614555] Key type ._llcrypt registered [10032.615822] Key type .llcrypt registered [10034.645898] Lustre: DEBUG MARKER: == conf-sanity test 62: start with disabled journal ====== 07:24:57 (1743506697) [10040.678780] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10040.702754] alg: No test for adler32 (adler32-zlib) [10041.916286] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [10042.234799] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [10042.237545] LNet: Accept secure, port 988 [10043.999383] Key type lgssc registered [10046.168725] Lustre: Echo OBD driver; http://www.lustre.org/ [10084.896704] Key type lgssc unregistered [10085.537148] LNet: 237517:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10085.548459] LNet: Removed LNI 192.168.206.17@tcp [10086.985397] Key type .llcrypt unregistered [10086.987258] Key type ._llcrypt unregistered [10111.265462] Key type ._llcrypt registered [10111.266505] Key type .llcrypt registered [10112.444093] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [10112.473680] alg: No test for adler32 (adler32-zlib) [10113.822782] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [10114.155988] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [10114.157816] LNet: Accept secure, port 988 [10115.871328] Key type lgssc registered [10117.516074] Lustre: Echo OBD driver; http://www.lustre.org/ [10181.682299] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10185.234236] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10199.555826] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10240.519345] Lustre: DEBUG MARKER: SKIP: conf-sanity test_63 skipping excluded test 63 [10242.480252] Lustre: DEBUG MARKER: == conf-sanity test 64: check lfs df --lazy ============== 07:28:26 (1743506906) [10263.215341] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10266.750805] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10279.470670] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10295.361530] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10297.440226] Lustre: Mounted lustre-client [10302.439275] Lustre: lustre-OST0001-osc-ffff966a00629000: Connection to lustre-OST0001 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [10307.300131] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [10317.615885] Lustre: Unmounted lustre-client [10349.670777] Key type lgssc unregistered [10350.178537] LNet: 244205:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10351.207161] LNet: Removed LNI 192.168.206.17@tcp [10352.378473] Key type .llcrypt unregistered [10352.379505] Key type ._llcrypt unregistered [10386.029129] Key type ._llcrypt registered [10386.030179] Key type .llcrypt registered [10387.378893] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [10387.402682] alg: No test for adler32 (adler32-zlib) [10388.664690] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [10389.029221] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [10389.031345] LNet: Accept secure, port 988 [10390.759314] Key type lgssc registered [10392.458916] Lustre: Echo OBD driver; http://www.lustre.org/ [10455.828581] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10459.533334] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10473.892457] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10507.548261] Lustre: DEBUG MARKER: == conf-sanity test 65: re-create the lost last_rcvd file when server mount ========================================================== 07:32:51 (1743507171) [10534.215713] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10538.138711] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10566.856073] Lustre: DEBUG MARKER: == conf-sanity test 66: replace nids ===================== 07:33:50 (1743507230) [10587.146444] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10590.678606] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10603.059802] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10604.708553] Lustre: Mounted lustre-client [10611.129608] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [10621.934090] Lustre: Unmounted lustre-client [10694.005862] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10697.986919] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10742.701257] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10748.928760] Lustre: Mounted lustre-client [10750.745686] Lustre: Unmounted lustre-client [10776.752515] Key type lgssc unregistered [10777.298221] LNet: 254277:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10778.342964] LNet: Removed LNI 192.168.206.17@tcp [10779.785467] Key type .llcrypt unregistered [10779.787309] Key type ._llcrypt unregistered [10808.516863] Key type ._llcrypt registered [10808.518208] Key type .llcrypt registered [10809.034579] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [10809.060547] alg: No test for adler32 (adler32-zlib) [10810.386799] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [10810.818142] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [10810.819916] LNet: Accept secure, port 988 [10812.615456] Key type lgssc registered [10814.954144] Lustre: Echo OBD driver; http://www.lustre.org/ [10876.268546] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10879.713548] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10894.025185] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10936.396315] Lustre: DEBUG MARKER: SKIP: conf-sanity test_67 skipping excluded test 67 [10938.802557] Lustre: DEBUG MARKER: == conf-sanity test 68: be able to reserve specific sequences in FLDB ========================================================== 07:40:02 (1743507602) [10958.244706] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10962.579210] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10975.118933] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10978.703410] Lustre: Mounted lustre-client [10980.609831] WARNING: CPU: 3 PID: 110103 at include/linux/backing-dev.h:291 ll_writepages+0x3dd/0x400 [lustre] [10980.611969] Modules linked in: lustre(O) osp(O) ofd(O) lod(O) mdt(O) mdd(O) mgs(O) lquota(O) lfsck(O) obdecho(O) mgc(O) mdc(O) lov(O) osc(O) lmv(O) fid(O) fld(O) ptlrpc_gss(O) ptlrpc(O) obdclass(O) ksocklnd(O) lnet(O) libcfs(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver intel_rapl_msr intel_rapl_common sb_edac rapl pcspkr i2c_piix4 squashfs crct10dif_pclmul crc32_pclmul crc32c_intel ata_generic ghash_clmulni_intel ata_piix serio_raw libata dm_mirror dm_region_hash dm_log dm_mod sha512_ssse3 sha512_generic [last unloaded: libcfs] [10980.620940] CPU: 3 PID: 110103 Comm: kworker/u8:2 Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #7 [10980.623241] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-1.fc39 04/01/2014 [10980.624926] Workqueue: writeback wb_workfn (flush-lustre-ffff966a00) [10980.626188] RIP: 0010:ll_writepages+0x3dd/0x400 [lustre] [10980.627259] Code: e8 b8 4b dc df 48 83 05 30 14 0f 00 01 85 c0 74 11 48 83 05 2c 14 0f 00 01 0f b6 43 24 e9 09 fd ff ff 48 83 05 23 14 0f 00 01 <0f> 0b 48 83 05 21 14 0f 00 01 48 83 05 21 14 0f 00 01 e9 5f ff ff [10980.630850] RSP: 0018:ffffa71eca487b58 EFLAGS: 00010202 [10980.631870] RAX: 0000000000000000 RBX: ffffa71eca487c40 RCX: 0000000000000001 [10980.633291] RDX: 0000000000000007 RSI: ffff9669ed3e10c8 RDI: 0000000000000206 [10980.634650] RBP: ffff9669ecee91d0 R08: 0000000000000000 R09: 0000000000000001 [10980.635934] R10: 000000000c82e86e R11: ffff9669fa311000 R12: ffff9669ecee9410 [10980.637224] R13: ffffffffffffffff R14: 0000000000000001 R15: 0000000000000000 [10980.638488] FS: 0000000000000000(0000) GS:ffff966a01e00000(0000) knlGS:0000000000000000 [10980.639962] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10980.640998] CR2: 0000558f00c1df44 CR3: 0000000024228002 CR4: 0000000000170ee0 [10980.642368] Call Trace: [10980.642871] ? show_regs.cold.9+0x22/0x2f [10980.643650] ? __warn+0xc8/0x150 [10980.644274] ? ll_writepages+0x3dd/0x400 [lustre] [10980.645224] ? report_bug+0x113/0x140 [10980.645886] ? do_error_trap+0xb6/0x130 [10980.646631] ? do_invalid_op+0x46/0x60 [10980.647387] ? ll_writepages+0x3dd/0x400 [lustre] [10980.649361] ? invalid_op+0x14/0x20 [10980.650530] ? ll_writepages+0x3dd/0x400 [lustre] [10980.651478] ? ll_writepages+0x3b8/0x400 [lustre] [10980.652674] do_writepages+0x7a/0x270 [10980.653812] __writeback_single_inode+0xb2/0x7d0 [10980.655670] ? do_raw_spin_unlock+0x75/0x190 [10980.656805] writeback_sb_inodes+0x239/0x7e0 [10980.657806] wb_writeback+0x1ae/0x7a0 [10980.658492] wb_workfn+0x1e5/0x950 [10980.659150] ? lock_acquire+0x146/0x3a0 [10980.660054] ? process_one_work+0x2c0/0x9b0 [10980.660859] process_one_work+0x3dd/0x9b0 [10980.661620] worker_thread+0x296/0x6e0 [10980.662376] ? rescuer_thread+0x570/0x570 [10980.663126] kthread+0x1d7/0x210 [10980.663885] ? set_kthread_struct+0x70/0x70 [10980.664831] ret_from_fork+0x1f/0x30 [10980.665945] ---[ end trace 4a77805e2a5b90c1 ]--- [10980.759767] Lustre: Unmounted lustre-client [11006.152702] Key type lgssc unregistered [11006.727174] LNet: 260532:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11007.784349] LNet: Removed LNI 192.168.206.17@tcp [11009.250545] Key type .llcrypt unregistered [11009.251881] Key type ._llcrypt unregistered [11026.797588] Key type ._llcrypt registered [11026.798570] Key type .llcrypt registered [11028.357609] Lustre: DEBUG MARKER: SKIP: conf-sanity test_69 skipping SLOW test 69 [11030.808635] Lustre: DEBUG MARKER: == conf-sanity test 70a: start MDT0, then OST, then MDT1 ========================================================== 07:41:34 (1743507694) [11036.869891] Key type .llcrypt unregistered [11036.871052] Key type ._llcrypt unregistered [11049.103951] Key type ._llcrypt registered [11049.104963] Key type .llcrypt registered [11049.557476] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [11049.581417] alg: No test for adler32 (adler32-zlib) [11050.892883] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [11051.320960] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [11051.323226] LNet: Accept secure, port 988 [11053.119373] Key type lgssc registered [11055.282850] Lustre: Echo OBD driver; http://www.lustre.org/ [11091.389254] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11105.274934] Lustre: Mounted lustre-client [11106.663490] Lustre: Unmounted lustre-client [11130.516930] Key type lgssc unregistered [11130.919931] LNet: 263798:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11131.942901] LNet: Removed LNI 192.168.206.17@tcp [11133.139691] Key type .llcrypt unregistered [11133.141416] Key type ._llcrypt unregistered [11151.598502] Key type ._llcrypt registered [11151.600086] Key type .llcrypt registered [11152.946790] Lustre: DEBUG MARKER: == conf-sanity test 70b: start OST, MDT1, MDT0 =========== 07:43:36 (1743507816) [11156.193323] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11156.214771] alg: No test for adler32 (adler32-zlib) [11157.423252] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [11157.713226] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [11157.716162] LNet: Accept secure, port 988 [11159.447297] Key type lgssc registered [11161.104057] Lustre: Echo OBD driver; http://www.lustre.org/ [11238.822669] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11257.827865] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11261.510724] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11263.142967] Lustre: Mounted lustre-client [11264.418909] Lustre: Unmounted lustre-client [11284.634381] Key type lgssc unregistered [11285.018929] LNet: 267092:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11286.054926] LNet: Removed LNI 192.168.206.17@tcp [11287.190789] Key type .llcrypt unregistered [11287.191994] Key type ._llcrypt unregistered [11304.150379] Key type ._llcrypt registered [11304.151363] Key type .llcrypt registered [11305.596145] Lustre: DEBUG MARKER: == conf-sanity test 70c: stop MDT0, mkdir fail, create remote dir fail ========================================================== 07:46:09 (1743507969) [11309.037815] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11309.057775] alg: No test for adler32 (adler32-zlib) [11310.194347] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [11310.462518] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [11310.464341] LNet: Accept secure, port 988 [11312.167362] Key type lgssc registered [11313.789306] Lustre: Echo OBD driver; http://www.lustre.org/ [11348.038417] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11351.514486] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11364.179740] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11366.140054] Lustre: Mounted lustre-client [11371.493566] Lustre: lustre-MDT0000-mdc-ffff9669fa384000: Connection to lustre-MDT0000 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [11375.028452] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [11375.049162] LustreError: 270008:0:(file.c:6007:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108 [11375.505950] LustreError: MGC192.168.206.117@tcp: Connection to MGS (at 192.168.206.117@tcp) was lost; in progress operations using this service will fail [11375.576380] Lustre: Unmounted lustre-client [11393.312440] Key type lgssc unregistered [11393.718853] LNet: 270419:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11394.789943] LNet: Removed LNI 192.168.206.17@tcp [11395.823424] Key type .llcrypt unregistered [11395.824638] Key type ._llcrypt unregistered [11411.782299] Key type ._llcrypt registered [11411.783644] Key type .llcrypt registered [11413.113710] Lustre: DEBUG MARKER: == conf-sanity test 70d: stop MDT1, mkdir succeed, create remote dir fail ========================================================== 07:47:56 (1743508076) [11416.922440] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11416.961311] alg: No test for adler32 (adler32-zlib) [11418.285055] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [11418.713450] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [11418.716906] LNet: Accept secure, port 988 [11420.487343] Key type lgssc registered [11422.519198] Lustre: Echo OBD driver; http://www.lustre.org/ [11457.296960] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11460.241685] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11472.013952] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11473.940505] Lustre: Mounted lustre-client [11482.989915] Lustre: setting import lustre-MDT0001_UUID INACTIVE by administrator request [11484.367150] Lustre: Unmounted lustre-client [11502.414372] Key type lgssc unregistered [11502.840960] LNet: 273755:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11503.910242] LNet: Removed LNI 192.168.206.17@tcp [11504.981996] Key type .llcrypt unregistered [11504.983143] Key type ._llcrypt unregistered [11522.261369] Key type ._llcrypt registered [11522.262365] Key type .llcrypt registered [11524.194399] Lustre: DEBUG MARKER: == conf-sanity test 70e: Sync-on-Cancel will be enabled by default on DNE ========================================================== 07:49:47 (1743508187) [11539.233989] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11539.268305] alg: No test for adler32 (adler32-zlib) [11540.599112] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [11541.040757] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [11541.042526] LNet: Accept secure, port 988 [11542.879384] Key type lgssc registered [11544.969782] Lustre: Echo OBD driver; http://www.lustre.org/ [11633.075914] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11634.657108] Lustre: Mounted lustre-client [11660.260460] Lustre: lustre-MDT0001-mdc-ffff9669f1049000: Connection to lustre-MDT0001 (at 192.168.206.117@tcp) was lost; in progress operations using this service will wait for recovery to complete [11666.555491] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request [11676.897410] Lustre: Unmounted lustre-client [11701.776491] Key type lgssc unregistered [11702.201869] LNet: 279223:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11703.270699] LNet: Removed LNI 192.168.206.17@tcp [11704.369364] Key type .llcrypt unregistered [11704.371037] Key type ._llcrypt unregistered [11721.354261] Key type ._llcrypt registered [11721.355168] Key type .llcrypt registered [11722.780542] Lustre: DEBUG MARKER: == conf-sanity test 71a: start MDT0 OST0, MDT1, OST1 ===== 07:53:06 (1743508386) [11724.673779] Lustre: DEBUG MARKER: SKIP: conf-sanity test_71a needs separate MGS/MDT [11726.601596] Lustre: DEBUG MARKER: == conf-sanity test 71b: start MDT1, OST0, MDT0, OST1 ==== 07:53:10 (1743508390) [11728.389094] Lustre: DEBUG MARKER: SKIP: conf-sanity test_71b needs separate MGS/MDT [11730.134924] Lustre: DEBUG MARKER: == conf-sanity test 71c: start OST0, OST1, MDT1, MDT0 ==== 07:53:14 (1743508394) [11731.910344] Lustre: DEBUG MARKER: SKIP: conf-sanity test_71c needs separate MGS/MDT [11733.970979] Lustre: DEBUG MARKER: == conf-sanity test 71d: start OST0, MDT1, MDT0, OST1 ==== 07:53:17 (1743508397) [11736.356922] Lustre: DEBUG MARKER: SKIP: conf-sanity test_71d needs separate MGS/MDT [11738.255276] Lustre: DEBUG MARKER: == conf-sanity test 71e: start OST0, MDT1, OST1, MDT0 ==== 07:53:22 (1743508402) [11740.500559] Lustre: DEBUG MARKER: SKIP: conf-sanity test_71e needs separate MGS/MDT [11742.863702] Lustre: DEBUG MARKER: == conf-sanity test 72: test fast symlink with extents flag enabled ========================================================== 07:53:26 (1743508406) [11749.296933] Key type .llcrypt unregistered [11749.298717] Key type ._llcrypt unregistered [11760.828723] Key type ._llcrypt registered [11760.829710] Key type .llcrypt registered [11761.257775] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [11761.285146] alg: No test for adler32 (adler32-zlib) [11762.757988] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [11763.189162] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [11763.191509] LNet: Accept secure, port 988 [11765.023314] Key type lgssc registered [11767.004719] Lustre: Echo OBD driver; http://www.lustre.org/ [11820.837548] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11824.645234] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11838.144287] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11839.807270] Lustre: Mounted lustre-client [11841.132319] Lustre: Unmounted lustre-client [11874.254357] Lustre: DEBUG MARKER: == conf-sanity test 73a: failnode to update from mountdata properly ========================================================== 07:55:37 (1743508537) [11893.794697] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11897.534393] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11908.798489] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11909.422068] Lustre: Mounted lustre-client [11910.921176] Lustre: Unmounted lustre-client [11939.363160] Lustre: DEBUG MARKER: == conf-sanity test 73b: Large failnode NID list in mountdata ========================================================== 07:56:42 (1743508602) [12011.955631] Lustre: DEBUG MARKER: == conf-sanity test 74: Test per-device adaptive timeout parameters ========================================================== 07:57:55 (1743508675) [12030.041980] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12034.044804] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12045.795685] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12046.434460] Lustre: Mounted lustre-client [12087.972971] Lustre: Mounted fs15246-client [12100.818263] Lustre: Unmounted fs15246-client [12118.752105] Lustre: Unmounted lustre-client [12150.195128] Key type lgssc unregistered [12150.606894] LNet: 292367:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12179.302537] LNet: Removed LNI 192.168.206.17@tcp [12180.740227] Key type .llcrypt unregistered [12180.742066] Key type ._llcrypt unregistered [12194.784693] Key type ._llcrypt registered [12194.787308] Key type .llcrypt registered [12197.053484] Lustre: DEBUG MARKER: == conf-sanity test 75: The order of --index should be irrelevant ========================================================== 08:00:59 (1743508859) [12203.144281] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12203.168950] alg: No test for adler32 (adler32-zlib) [12204.431040] Lustre: Lustre: Build Version: 2.16.52_73_g6bb624e [12204.834704] LNet: Added LNI 192.168.206.17@tcp [8/256/0/180] [12204.837265] LNet: Accept secure, port 988 [12206.615386] Key type lgssc registered [12208.109648] Lustre: Echo OBD driver; http://www.lustre.org/ [12301.270583] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12304.701074] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12317.331213] Lustre: DEBUG MARKER: oleg617-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12402.629230] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11817 sec ========= 08:04:26 (1743509066) [12404.931846] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 08:04:28 (1743509068) === [12409.813110] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 08:04:33 (1743509073) === [12432.907432] Key type lgssc unregistered [12433.387559] LNet: 301125:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12433.399412] LNet: Removed LNI 192.168.206.17@tcp [12434.817612] Key type .llcrypt unregistered [12434.818639] Key type ._llcrypt unregistered