[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0.0 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-2.fc40 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 380051515 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5410-0x000f541f] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5220 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1D6F 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1C0B 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001BCB (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1C7F 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1D0F 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1D47 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1c0b-0xbffe1c7e] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1c0a] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1c7f-0xbffe1d0e] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1d0f-0xbffe1d46] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1d47-0xbffe1d6e] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829700K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002265] x2apic enabled [ 0.003004] Switched APIC routing to physical x2apic. [ 0.004006] kvm-guest: setup PV IPIs [ 0.006000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.006015] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.007006] pid_max: default: 32768 minimum: 301 [ 0.008124] LSM: Security Framework initializing [ 0.009027] Yama: becoming mindful. [ 0.010019] SELinux: Initializing. [ 0.010765] *** VALIDATE selinux *** [ 0.017623] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.021426] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.022083] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.023065] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.024055] *** VALIDATE tmpfs *** [ 0.025338] *** VALIDATE proc *** [ 0.026139] *** VALIDATE cgroup *** [ 0.027004] *** VALIDATE cgroup2 *** [ 0.028187] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.029094] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.030002] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.032008] Spectre V2 : User space: Vulnerable [ 0.032892] Speculative Store Bypass: Vulnerable [ 0.035291] debug: unmapping init [mem 0xffffffffb6a59000-0xffffffffb6a60fff] [ 0.037803] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.038408] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.039009] ... version: 2 [ 0.039784] ... bit width: 48 [ 0.040005] ... generic registers: 4 [ 0.040736] ... value mask: 0000ffffffffffff [ 0.041005] ... max period: 00007fffffffffff [ 0.041997] ... fixed-purpose events: 3 [ 0.042005] ... event mask: 000000070000000f [ 0.043175] rcu: Hierarchical SRCU implementation. [ 0.045085] smp: Bringing up secondary CPUs ... [ 0.046378] x86: Booting SMP configuration: [ 0.047014] .... node #0, CPUs: #1 #2 #3 [ 0.049396] smp: Brought up 1 node, 4 CPUs [ 0.050780] smpboot: Max logical packages: 1 [ 0.051007] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.135015] node 0 deferred pages initialised in 83ms [ 0.137156] devtmpfs: initialized [ 0.138156] x86/mm: Memory block size: 128MB [ 0.140272] gcov: version magic: 0x41383552 [ 0.141543] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.142057] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.143155] pinctrl core: initialized pinctrl subsystem [ 0.144099] [ 0.144484] ************************************************************* [ 0.145006] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.146005] ** ** [ 0.147006] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.148005] ** ** [ 0.149008] ** This means that this kernel is built to expose internal ** [ 0.150007] ** IOMMU data structures, which may compromise security on ** [ 0.151007] ** your system. ** [ 0.152018] ** ** [ 0.153006] ** If you see this message and you are not debugging the ** [ 0.154005] ** kernel, report this immediately to your vendor! ** [ 0.155005] ** ** [ 0.156005] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.157005] ************************************************************* [ 0.158487] NET: Registered protocol family 16 [ 0.159331] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.160038] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.161027] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.162265] cpuidle: using governor menu [ 0.163529] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.165326] PCI: Using configuration type 1 for base access [ 0.167089] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.174076] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.175018] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.176107] cryptd: max_cpu_qlen set to 1000 [ 0.177207] ACPI: Added _OSI(Module Device) [ 0.178021] ACPI: Added _OSI(Processor Device) [ 0.179013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.180010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.182915] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.187373] ACPI: Interpreter enabled [ 0.188033] ACPI: PM: (supports S0 S3 S4 S5) [ 0.188965] ACPI: Using IOAPIC for interrupt routing [ 0.190063] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.192230] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.198959] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.200017] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.202008] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.204048] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.207037] acpiphp: Slot [2] registered [ 0.207948] acpiphp: Slot [5] registered [ 0.209055] acpiphp: Slot [6] registered [ 0.209936] acpiphp: Slot [3] registered [ 0.211043] acpiphp: Slot [4] registered [ 0.211889] acpiphp: Slot [7] registered [ 0.212077] acpiphp: Slot [8] registered [ 0.212965] acpiphp: Slot [9] registered [ 0.214058] acpiphp: Slot [10] registered [ 0.214929] acpiphp: Slot [11] registered [ 0.216044] acpiphp: Slot [12] registered [ 0.216912] acpiphp: Slot [13] registered [ 0.217083] acpiphp: Slot [14] registered [ 0.217925] acpiphp: Slot [15] registered [ 0.219044] acpiphp: Slot [16] registered [ 0.219944] acpiphp: Slot [17] registered [ 0.221043] acpiphp: Slot [18] registered [ 0.221908] acpiphp: Slot [19] registered [ 0.223042] acpiphp: Slot [20] registered [ 0.223849] acpiphp: Slot [21] registered [ 0.224058] acpiphp: Slot [22] registered [ 0.224908] acpiphp: Slot [23] registered [ 0.226050] acpiphp: Slot [24] registered [ 0.226902] acpiphp: Slot [25] registered [ 0.228053] acpiphp: Slot [26] registered [ 0.228946] acpiphp: Slot [27] registered [ 0.229041] acpiphp: Slot [28] registered [ 0.230022] acpiphp: Slot [29] registered [ 0.230995] acpiphp: Slot [30] registered [ 0.232049] acpiphp: Slot [31] registered [ 0.232916] PCI host bridge to bus 0000:00 [ 0.234010] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.236008] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.237007] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.238008] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.240008] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x38007fffffff window] [ 0.242009] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.243117] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.244654] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.246785] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.251443] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.255006] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.257007] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.258006] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.259007] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.261375] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.262550] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.264021] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.266452] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.268007] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.274771] pci 0000:00:02.0: reg 0x20: [mem 0x380000000000-0x380000003fff 64bit pref] [ 0.278787] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.281649] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.289008] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.293010] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.302010] pci 0000:00:05.0: reg 0x20: [mem 0x380000004000-0x380000007fff 64bit pref] [ 0.310405] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.313941] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.318009] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.325012] pci 0000:00:06.0: reg 0x20: [mem 0x380000008000-0x38000000bfff 64bit pref] [ 0.335078] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.337287] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.339291] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.341264] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.344104] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.347032] iommu: Default domain type: Passthrough [ 0.348320] SCSI subsystem initialized [ 0.349076] ACPI: bus type USB registered [ 0.350059] usbcore: registered new interface driver usbfs [ 0.352031] usbcore: registered new interface driver hub [ 0.353047] usbcore: registered new device driver usb [ 0.354088] pps_core: LinuxPPS API ver. 1 registered [ 0.355006] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.357019] PTP clock support registered [ 0.359011] EDAC MC: Ver: 3.0.0 [ 0.360215] PCI: Using ACPI for IRQ routing [ 0.361459] NetLabel: Initializing [ 0.363012] NetLabel: domain hash size = 128 [ 0.364008] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.366042] NetLabel: unlabeled traffic allowed by default [ 0.367102] vgaarb: loaded [ 0.368258] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.369006] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.375302] clocksource: Switched to clocksource kvm-clock [ 0.468224] VFS: Disk quotas dquot_6.6.0 [ 0.469410] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.471074] *** VALIDATE ramfs *** [ 0.471922] *** VALIDATE hugetlbfs *** [ 0.473032] pnp: PnP ACPI init [ 0.474962] pnp: PnP ACPI: found 6 devices [ 0.489181] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.491712] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.493436] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.495034] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.496989] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.498723] pci_bus 0000:00: resource 8 [mem 0x380000000000-0x38007fffffff window] [ 0.500790] NET: Registered protocol family 2 [ 0.502757] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.506532] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.509022] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.513758] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.516744] TCP: Hash tables configured (established 65536 bind 65536) [ 0.519375] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.521688] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.523735] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.525796] NET: Registered protocol family 1 [ 0.527715] RPC: Registered named UNIX socket transport module. [ 0.529259] RPC: Registered udp transport module. [ 0.530441] RPC: Registered tcp transport module. [ 0.531744] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.533341] NET: Registered protocol family 44 [ 0.534588] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.536024] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.537693] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.539282] PCI: CLS 0 bytes, default 64 [ 0.540593] Unpacking initramfs... [ 1.947891] debug: unmapping init [mem 0xffff89fe7cc64000-0xffff89fe7ffcffff] [ 1.954410] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 1.956542] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 1.960410] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.419069] Initialise system trusted keyrings [ 2.420662] Key type blacklist registered [ 2.422758] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.432524] zbud: loaded [ 2.436332] *** VALIDATE nfs *** [ 2.437546] *** VALIDATE nfs4 *** [ 2.439472] pstore: using deflate compression [ 2.443271] Platform Keyring initialized [ 2.540733] NET: Registered protocol family 38 [ 2.542633] Key type asymmetric registered [ 2.544124] Asymmetric key parser 'x509' registered [ 2.545812] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.548384] io scheduler mq-deadline registered [ 2.549789] io scheduler kyber registered [ 2.551371] io scheduler bfq registered [ 2.552914] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.555418] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.557979] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.560604] ACPI: Power Button [PWRF] [ 2.654790] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.756274] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.867107] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.896077] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.928565] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.935168] Non-volatile memory driver v1.3 [ 2.936662] Linux agpgart interface v0.103 [ 2.964103] virtio_blk virtio1: [vda] 132808 512-byte logical blocks (68.0 MB/64.8 MiB) [ 2.966844] vda: detected capacity change from 0 to 67997696 [ 2.979174] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.981329] vdb: detected capacity change from 0 to 1073741824 [ 2.986953] libphy: Fixed MDIO Bus: probed [ 2.990104] usbcore: registered new interface driver usbserial_generic [ 2.991675] usbserial: USB Serial support registered for generic [ 2.992902] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.995768] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.996669] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.998177] mousedev: PS/2 mouse device common for all mice [ 3.000432] rtc_cmos 00:05: RTC can wake from S4 [ 3.000577] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.003242] rtc_cmos 00:05: registered as rtc0 [ 3.007783] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.010237] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.012312] intel_pstate: CPU model not supported [ 3.018558] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.022714] hid: raw HID events driver (C) Jiri Kosina [ 3.024780] usbcore: registered new interface driver usbhid [ 3.025753] usbhid: USB HID core driver [ 3.026672] drop_monitor: Initializing network drop monitor service [ 3.029091] Initializing XFRM netlink socket [ 3.031335] NET: Registered protocol family 10 [ 3.034360] Segment Routing with IPv6 [ 3.035679] NET: Registered protocol family 17 [ 3.037476] mpls_gso: MPLS GSO support [ 3.041442] RAS: Correctable Errors collector initialized. [ 3.042856] AVX version of gcm_enc/dec engaged. [ 3.043813] AES CTR mode by8 optimization enabled [ 3.102527] sched_clock: Marking stable (3102505400, 0)->(3775852548, -673347148) [ 3.105164] registered taskstats version 1 [ 3.106502] Loading compiled-in X.509 certificates [ 3.107986] zswap: loaded using pool lzo/zbud [ 3.129123] Key type big_key registered [ 3.139082] Key type encrypted registered [ 3.140519] ima: No TPM chip found, activating TPM-bypass! [ 3.142366] ima: Allocated hash algorithm: sha1 [ 3.143889] ima: No architecture policies found [ 3.145508] evm: Initialising EVM extended attributes: [ 3.147243] evm: security.selinux [ 3.148387] evm: security.ima [ 3.149415] evm: security.capability [ 3.150626] evm: HMAC attrs: 0x1 [ 3.152771] rtc_cmos 00:05: setting system clock to 2025-08-19 20:54:25 UTC (1755636865) [ 3.158996] debug: unmapping init [mem 0xffffffffb7a03000-0xffffffffb7bfffff] [ 3.162092] debug: unmapping init [mem 0xffffffffb6782000-0xffffffffb6a58fff] [ 3.171111] Write protecting the kernel read-only data: 28672k [ 3.177293] debug: unmapping init [mem 0xffffffffb4e03000-0xffffffffb4ffffff] [ 3.182269] debug: unmapping init [mem 0xffffffffb5714000-0xffffffffb57fffff] [ 3.228256] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.241993] systemd[1]: Detected virtualization kvm. [ 3.243649] systemd[1]: Detected architecture x86-64. [ 3.245566] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.276937] systemd[1]: No hostname configured. [ 3.280135] systemd[1]: Set hostname to . [ 3.282562] random: systemd: uninitialized urandom read (16 bytes read) [ 3.285840] systemd[1]: Initializing machine ID from random generator. [ 3.338760] random: ln: uninitialized urandom read (6 bytes read) [ 3.461804] random: systemd: uninitialized urandom read (16 bytes read) [ 3.464388] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.474129] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.478262] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Swap. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Slices. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Create Volatile Files and Directories... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.219460] device-mapper: uevent: version 1.0.3 [ 4.221444] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.197185] virtio_net virtio0 ens2: renamed from eth0 [ 5.366416] scsi host0: ata_piix [ 5.384283] scsi host1: ata_piix [ 5.386849] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 5.390513] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 9.756371] random: crng init done [ 9.757827] random: 7 urandom warning(s) missed due to ratelimiting [ 9.954093] hrtimer: interrupt took 36145952 ns [ 10.316854] dracut-initqueue[585]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 13.275471] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 15.369971] printk: systemd: 26 output lines suppressed due to ratelimiting [ 15.673944] SELinux: Disabled at runtime. [ 15.729584] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 15.752941] systemd[1]: Detected virtualization kvm. [ 15.755015] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 16.799150] systemd[1]: initrd-switch-root.service: Succeeded. [ 16.804135] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 16.815345] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 16.822910] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 16.827081] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 16.838725] systemd[1]: Starting Journal Service... Starting Journal Service... [ 16.849427] systemd[1]: Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-getty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Kernel Debug File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Mounting POSIX Message Queue File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. Activating swap /dev/disk/by-label/SWAP... Mounting Huge Pages File System... [ OK ] Created slice User and Session Slice. [ OK ] Reached target RPC Port Mapper. Starting Create list of required st…ce nodes for the current kernel... [ 17.086757] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Reached target rpc_pipefs.target. Starting Apply Kernel Variables... [ OK ] Listening on initctl Compatibility Named Pipe. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target Slices. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Stopped target Initrd File Systems. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ 18.327820] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /mnt. [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 19.270904] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 19.271640] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 19.921332] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 19.935932] EDAC sbridge: Ver: 1.1.2 [ 22.009868] Key type dns_resolver registered [ 22.391393] NFS: Registering the id_resolver key type [ 22.393487] Key type id_resolver registered [ 22.395267] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg134-client login: [ 55.098875] libcfs: loading out-of-tree module taints kernel. [ 55.111098] Key type ._llcrypt registered [ 55.112771] Key type .llcrypt registered [ 55.362299] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 55.367741] alg: No test for adler32 (adler32-zlib) [ 56.349419] Lustre: Lustre: Build Version: 2.16.57_62_g5cfb5f1 [ 56.649742] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 58.248222] Key type lgssc registered [ 58.834882] Lustre: Echo OBD driver; http://www.lustre.org/ [ 102.731227] Lustre: Mounted lustre-client [ 105.032895] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 115.410419] Lustre: DEBUG MARKER: oleg134-client.virtnet: executing check_logdir /tmp/testlogs/ [ 116.871594] Lustre: DEBUG MARKER: oleg134-client.virtnet: executing yml_node [ 118.548664] Lustre: DEBUG MARKER: Client: 2.16.57.62 [ 119.433833] Lustre: DEBUG MARKER: MDS: 2.16.57.62 [ 120.264796] Lustre: DEBUG MARKER: OSS: 2.16.57.62 [ 120.846953] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Tue Aug 19 16:56:22 EDT 2025 [ 125.911262] LustreError: 7917:0:(lov_obd.c:784:lov_cleanup()) lustre-clilov-ffff89fed1f2a800: lov tgt 0 not cleaned! deathrow=0, lovrc=1 [ 125.917337] LustreError: 7917:0:(obd_class.h:479:obd_check_dev()) Device 5 not setup [ 125.927849] Lustre: Unmounted lustre-client [ 135.611909] Key type lgssc unregistered [ 135.740394] LNet: 8361:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 135.743558] LNetError: 8361:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 135.749457] LNet: Removed LNI 192.168.201.34@tcp [ 136.021112] Key type .llcrypt unregistered [ 136.022729] Key type ._llcrypt unregistered [ 144.292334] Key type ._llcrypt registered [ 144.293369] Key type .llcrypt registered [ 144.353194] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 144.358820] alg: No test for adler32 (adler32-zlib) [ 145.158546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 145.186659] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 145.197045] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show [ 146.452598] Lustre: DEBUG MARKER: excepting tests: 253 254 410 [ 146.478978] LNet: 9225:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 146.480875] LNetError: 9225:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 146.488403] LNet: Removed LNI 192.168.201.34@tcp [ 146.730374] Key type .llcrypt unregistered [ 146.731963] Key type ._llcrypt unregistered [ 148.593533] Key type ._llcrypt registered [ 148.594651] Key type .llcrypt registered [ 149.042667] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 16:56:50 (1755637010) [ 149.140186] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 149.145733] alg: No test for adler32 (adler32-zlib) [ 149.943360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 149.975691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 149.984733] LNet: 9649:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 149.994723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 150.014698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 150.022797] LNet: 9657:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 152.162887] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 16:56:54 (1755637014) [ 152.189636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 152.200918] LNet: 10227:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 152.210778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 152.244532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp [ 154.362986] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 16:56:56 (1755637016) [ 154.390022] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 154.398812] LNet: 10811:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 154.408778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 154.441986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 2.2.2.2@o2ib [ 156.550030] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 16:56:58 (1755637018) [ 156.577861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 156.586571] LNet: 11397:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 156.595822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 156.625847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib [ 158.674798] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 16:57:00 (1755637020) [ 158.702267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 158.723095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 158.759714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp [ 158.890748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp [ 160.883423] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 16:57:02 (1755637022) [ 160.909235] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 160.918969] LNet: 12568:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 160.921584] LNet: 12568:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 160.929803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 160.958918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 161.091173] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 161.219111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib [ 163.230459] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 16:57:05 (1755637025) [ 163.258056] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 163.275926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 163.305502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi,[6-12/4]@bxi3f [ 165.420484] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 16:57:07 (1755637027) [ 165.449884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 165.468658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 165.504822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp [ 165.519387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 165.648879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib [ 165.664130] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 165.796618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp [ 165.808487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 165.925362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib [ 165.937722] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 166.050232] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib [ 166.065509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 166.203874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni [ 166.214402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 166.347805] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni [ 166.364466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 166.492424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi [ 166.506385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 166.646856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi [ 166.659604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 166.798109] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f [ 166.811745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 166.941570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f --nid [8-12]@bxi3f [ 166.955486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 167.088110] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi,[3-7]@bxi3f [ 167.103134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 167.237820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 167.252267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 169.397432] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 16:57:11 (1755637031) [ 169.425073] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 169.433629] LNet: 14428:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 169.435752] LNet: 14428:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 169.444276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 169.473187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp [ 169.484502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp [ 171.564878] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 16:57:13 (1755637033) [ 171.604244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 171.622083] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 171.650062] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 171.663601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 173.816508] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 16:57:15 (1755637035) [ 173.845752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 173.863270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 173.899880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp [ 173.913665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp [ 176.056092] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 16:57:17 (1755637037) [ 176.084456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 176.101552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 176.137475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 176.152294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 178.264058] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 16:57:20 (1755637040) [ 178.302804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 178.319465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 178.351729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp [ 178.367189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp [ 180.601785] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 16:57:22 (1755637042) [ 180.645355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 180.663700] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 180.696622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 180.715315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 182.922968] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 16:57:24 (1755637044) [ 182.950900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 182.967892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 183.003295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 183.152807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 185.264778] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 16:57:27 (1755637047) [ 185.292589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 185.309131] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 185.384311] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 185.457710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 187.642723] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 16:57:29 (1755637049) [ 187.672935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 187.682616] LNet: 19154:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 187.685194] LNet: 19154:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 7 previous similar messages [ 187.694539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 187.736790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 187.821732] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 189.931958] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 16:57:31 (1755637051) [ 189.963847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 189.980527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 190.139623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 190.236802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 192.500395] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 16:57:34 (1755637054) [ 192.540663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 192.561944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 192.598593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 192.620293] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 195.027044] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 16:57:36 (1755637056) [ 195.058605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 195.080349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 195.119749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 195.142563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 197.468206] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 16:57:39 (1755637059) [ 197.496836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 197.515312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 197.549147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 19@gni [ 199.786455] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 16:57:41 (1755637061) [ 199.817589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 199.835134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 199.866159] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib [ 201.958331] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 16:57:43 (1755637063) [ 201.993713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 202.015242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 202.054367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni [ 202.194642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni [ 204.455251] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 16:57:46 (1755637066) [ 204.482500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 204.502184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 204.537562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni [ 204.552513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 22@gni --nid 26@gni [ 206.769477] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 16:57:48 (1755637068) [ 206.804240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 206.823627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 206.868327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni [ 206.886349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni [ 209.088860] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 16:57:50 (1755637070) [ 209.116140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 209.131668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 209.163608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni [ 209.178016] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp [ 211.267519] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 16:57:53 (1755637073) [ 211.304195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 211.320372] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 211.353724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 211.373642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 213.553980] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 16:57:55 (1755637075) [ 213.582259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 213.599882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 213.627990] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim [ 213.652531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp [ 213.662082] LNetError: 25733:0:(peer.c:2209:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 213.677567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --force [ 213.691495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show --nid 1.1.1.1@tcp [ 215.681400] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 16:57:57 (1755637077) [ 215.707136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 215.721169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 215.755119] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.PF6JV4YcGM [ 215.781255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.PF6JV4YcGM [ 215.800963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.PF6JV4YcGM [ 217.904502] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 16:57:59 (1755637079) [ 217.938049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 217.956602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 217.984889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp [ 218.001530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp [ 218.021850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer list [ 220.139270] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 16:58:02 (1755637082) [ 220.177802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 220.187895] LNet: 27483:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 220.191520] LNet: 27483:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 13 previous similar messages [ 220.203899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 220.228323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid foobar [ 220.243356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid foobar [ 220.259189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@o2ib [ 220.273264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --nid 1.1.1.1@tcp [ 220.285424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --nid 1.1.1.1@tcp [ 220.299597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add [ 220.312425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 220.326507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp [ 220.341447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@foo [ 220.354094] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@tcp [ 220.369139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@o2ib [ 220.383939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@gni [ 220.397735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni [ 220.412336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni [ 220.429051] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp [ 220.446122] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp [ 220.465049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp [ 220.484073] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 220.502213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp --gateway 1@gni [ 220.512601] LNetError: 27531:0:(router.c:632:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni [ 222.553878] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 16:58:04 (1755637084) [ 222.588649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 222.607796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 222.648360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 224.854128] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 16:58:06 (1755637086) [ 225.475954] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_100 Need at least 2 remote nodes found oleg134-server [ 226.066439] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 16:58:07 (1755637087) [ 226.102055] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.120840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.148556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 226.158188] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 226.691894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp1 --gateway 192.168.201.[35-39]@tcp [ 226.701697] LNet: 29242:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 226.710207] LNetError: 29035:0:(router.c:400:lnet_router_discovery_ping_reply()) Peer 192.168.201.35@tcp is being used as a gateway but routing feature is not turned on [ 226.714026] LNetError: 29035:0:(lib-lnet.h:1327:lnet_set_route_aliveness()) route to tcp1 through 192.168.201.35@tcp has gone from up to down [ 226.716950] LNetError: 29244:0:(router.c:334:lnet_is_route_alive()) peer 192.168.201.35@tcp is being used as a gateway but routing feature is not turned on [ 228.805416] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 16:58:10 (1755637090) [ 228.824356] LNetError: 29825:0:(router.c:334:lnet_is_route_alive()) peer 192.168.201.35@tcp is being used as a gateway but routing feature is not turned on [ 229.420971] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_102 Need at least 2 remote nodes found oleg134-server [ 229.792417] LNetError: 29041:0:(lib-lnet.h:1327:lnet_set_route_aliveness()) route to tcp1 through 192.168.201.37@tcp has gone from up to down [ 229.998795] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 16:58:11 (1755637091) [ 230.029532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 230.039373] LNetError: 30171:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 230.049644] LNet: Removed LNI 192.168.201.34@tcp [ 230.061195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 230.081679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 230.091187] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 230.307909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp103 --gateway 192.168.201.[35-40/2]@tcp [ 230.318574] LNet: 30290:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 230.321822] LNet: 30290:0:(router.c:737:lnet_add_route()) Skipped 4 previous similar messages [ 230.327372] LNetError: 30180:0:(router.c:400:lnet_router_discovery_ping_reply()) Peer 192.168.201.35@tcp is being used as a gateway but routing feature is not turned on [ 230.331583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route del --net tcp103 --gateway 192.168.201.[35-40/2]@tcp [ 230.333471] LNetError: 30180:0:(router.c:400:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message [ 232.447902] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 16:58:14 (1755637094) [ 232.478965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 233.504643] LNetError: 30877:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 233.515840] LNet: Removed LNI 192.168.201.34@tcp [ 233.524349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 233.556788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking -1 [ 233.580808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 233.595977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 233.625632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 233.646929] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 233.661102] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 233.685438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 0 [ 233.718684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 233.734725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 233.762283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 233.794763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 233.812092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 233.835930] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 1 [ 233.868721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 233.885880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 233.916049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 233.946807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 233.962617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 233.989435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 2 [ 234.023627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 234.039226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 234.066507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 234.100236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 234.114495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 234.137250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 234.172525] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 234.186671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 234.210946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 234.243879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 234.257645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 234.278325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 4 [ 234.306813] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 234.324573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 234.353891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 236.327194] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 16:58:18 (1755637098) [ 236.935822] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_105 Need at least 2 remote nodes found oleg134-server [ 237.470898] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 16:58:19 (1755637099) [ 238.071921] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_106 Need at least 2 remote nodes found oleg134-server [ 238.612555] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 16:58:20 (1755637100) [ 238.879714] Key type .llcrypt unregistered [ 238.881338] Key type ._llcrypt unregistered [ 240.674300] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 240.730186] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 240.733243] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 240.736110] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 240.838665] Key type ._llcrypt registered [ 240.840264] Key type .llcrypt registered [ 240.910651] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 240.917568] alg: No test for adler32 (adler32-zlib) [ 241.719231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 241.742840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 241.754849] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 241.981746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 241.990153] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 242.679822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 242.689702] LNet: Removed LNI 10.1.2.3@tcp [ 242.728794] LNet: 32966:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 242.731980] LNetError: 32966:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 242.741369] LNet: Removed LNI 192.168.201.34@tcp [ 242.974352] Key type .llcrypt unregistered [ 242.975554] Key type ._llcrypt unregistered [ 246.638032] Key type ._llcrypt registered [ 246.639484] Key type .llcrypt registered [ 247.100620] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 16:58:28 (1755637108) [ 247.283288] Key type .llcrypt unregistered [ 247.284473] Key type ._llcrypt unregistered [ 249.110366] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 249.173327] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 249.182933] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 249.185259] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 249.288556] Key type ._llcrypt registered [ 249.290634] Key type .llcrypt registered [ 249.364581] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 249.371126] alg: No test for adler32 (adler32-zlib) [ 250.168447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 250.187463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 250.196507] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 250.432325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 250.440656] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 251.270690] LNet: 34419:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 251.273201] LNetError: 34419:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 251.276579] LNet: Removed LNI 192.168.201.34@tcp [ 251.282302] LNet: Removed LNI 10.1.2.3@tcp [ 251.499424] Key type .llcrypt unregistered [ 251.501092] Key type ._llcrypt unregistered [ 254.887124] Key type ._llcrypt registered [ 254.888159] Key type .llcrypt registered [ 255.317641] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 16:58:37 (1755637117) [ 255.479216] Key type .llcrypt unregistered [ 255.480650] Key type ._llcrypt unregistered [ 257.289124] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 257.360777] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 257.362263] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 257.364176] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 257.488735] Key type ._llcrypt registered [ 257.490341] Key type .llcrypt registered [ 257.554807] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 257.561322] alg: No test for adler32 (adler32-zlib) [ 258.363134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 258.383897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 258.394586] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 259.021805] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 259.031025] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 259.692255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 259.701490] LNet: Removed LNI 10.1.2.3@tcp [ 259.708950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg:0 [ 259.723560] LNet: Removed LNI 10.1.2.31@tcp [ 259.729775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 259.738887] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 259.984417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 259.993195] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 260.306331] LNet: 36292:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 260.309199] LNetError: 36292:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface test1pg not found [ 260.311756] LNet: Removed LNI 10.1.2.3@tcp [ 260.320301] LNet: Removed LNI 10.1.2.31@tcp [ 260.542045] Key type .llcrypt unregistered [ 260.543349] Key type ._llcrypt unregistered [ 264.189608] Key type ._llcrypt registered [ 264.191191] Key type .llcrypt registered [ 264.710601] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 16:58:46 (1755637126) [ 264.828780] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 264.836762] alg: No test for adler32 (adler32-zlib) [ 265.650239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 265.673485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 265.685289] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 265.971114] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 265.981446] LNet: 37236:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 265.985632] LNetError: 37236:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 265.994397] LNet: Removed LNI 192.168.201.34@tcp [ 266.002798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 266.027403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --nid 192.168.201.34@tcp [ 266.040526] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 266.294431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --nid 192.168.201.34@tcp [ 266.308443] LNet: Removed LNI 192.168.201.34@tcp [ 266.573566] Key type .llcrypt unregistered [ 266.575420] Key type ._llcrypt unregistered [ 270.249821] Key type ._llcrypt registered [ 270.251020] Key type .llcrypt registered [ 270.694460] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 16:58:52 (1755637132) [ 271.271532] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_111 Need at least 2 remote nodes found oleg134-server [ 271.798253] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 16:58:53 (1755637133) [ 271.947518] Key type .llcrypt unregistered [ 271.948757] Key type ._llcrypt unregistered [ 273.719533] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 273.773206] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 273.777700] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 273.780867] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 273.893979] Key type ._llcrypt registered [ 273.895586] Key type .llcrypt registered [ 273.969848] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 273.977329] alg: No test for adler32 (adler32-zlib) [ 274.780809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 274.806095] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 275.039320] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180] [ 275.780292] LNet: 39161:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 275.783923] LNetError: 39161:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 275.786615] LNet: Removed LNI 192.168.201.34@tcp1 [ 275.792402] LNet: Removed LNI 10.1.2.3@tcp2 [ 276.022414] Key type .llcrypt unregistered [ 276.023541] Key type ._llcrypt unregistered [ 279.557188] Key type ._llcrypt registered [ 279.558467] Key type .llcrypt registered [ 280.019126] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 16:59:01 (1755637141) [ 280.143030] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 280.149422] alg: No test for adler32 (adler32-zlib) [ 280.955658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 280.978086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 280.988820] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 281.266158] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --ip2net tcp 192.168.201.34 [ 281.275947] LNet: Removed LNI 192.168.201.34@tcp [ 283.426375] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 16:59:05 (1755637145) [ 283.458892] LNet: 40679:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 283.463101] LNetError: 40679:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 283.472498] LNet: Removed LNI 192.168.201.34@tcp [ 283.735569] Key type .llcrypt unregistered [ 283.737376] Key type ._llcrypt unregistered [ 285.668612] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 285.781238] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 285.783850] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 285.786505] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 285.933463] Key type ._llcrypt registered [ 285.934795] Key type .llcrypt registered [ 285.997910] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 286.004386] alg: No test for adler32 (adler32-zlib) [ 286.962562] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 289.430788] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 16:59:11 (1755637151) [ 289.484989] LNet: 41558:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 289.489443] LNetError: 41558:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface test1pg not found [ 289.497723] LNet: Removed LNI 10.1.2.3@tcp [ 289.753651] Key type .llcrypt unregistered [ 289.755439] Key type ._llcrypt unregistered [ 291.749531] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 291.841489] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 291.844215] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 291.846777] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 291.976672] Key type ._llcrypt registered [ 291.978112] Key type .llcrypt registered [ 292.042766] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 292.051262] alg: No test for adler32 (adler32-zlib) [ 292.864443] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 295.026110] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 16:59:16 (1755637156) [ 295.065433] LNet: 42420:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 295.069376] LNetError: 42420:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface test1pg not found [ 295.077565] LNet: Removed LNI 10.1.2.3@tcp [ 295.336580] Key type .llcrypt unregistered [ 295.338527] Key type ._llcrypt unregistered [ 297.257137] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 297.363558] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 297.366197] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 297.369217] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 297.507701] Key type ._llcrypt registered [ 297.510338] Key type .llcrypt registered [ 297.584409] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 297.592691] alg: No test for adler32 (adler32-zlib) [ 298.422715] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 300.795669] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 16:59:22 (1755637162) [ 300.827295] LNet: 43284:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 300.830196] LNetError: 43284:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface test1pg not found [ 300.841645] LNet: Removed LNI 10.1.2.3@tcp [ 301.118483] Key type .llcrypt unregistered [ 301.120328] Key type ._llcrypt unregistered [ 303.159121] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 303.258870] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 303.261341] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 303.264073] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 303.388969] Key type ._llcrypt registered [ 303.390112] Key type .llcrypt registered [ 303.452289] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 303.458519] alg: No test for adler32 (adler32-zlib) [ 304.274847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 304.306493] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 307.469643] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 16:59:29 (1755637169) [ 307.620881] LNet: 44489:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 307.626338] LNetError: 44489:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface test1pg not found [ 307.636840] LNet: Removed LNI 10.1.2.3@tcp [ 307.893440] Key type .llcrypt unregistered [ 307.895416] Key type ._llcrypt unregistered [ 310.148958] Key type ._llcrypt registered [ 310.150669] Key type .llcrypt registered [ 310.245330] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 310.252095] alg: No test for adler32 (adler32-zlib) [ 311.077631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 311.091159] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 314.397433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 314.699337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_interrupt [ 314.719094] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_interrupt [ 314.737055] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 314.747816] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 314.851993] LNet: 44901:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 314.883307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 314.903223] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 314.924996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 315.111473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_dropped [ 315.127125] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_dropped [ 315.143078] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 315.282783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 315.298889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 315.314818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 315.513721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_aborted [ 315.531944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_aborted [ 315.550194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 315.561279] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 315.564232] LNet: Skipped 1 previous similar message [ 315.674346] LNet: 45071:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 315.678721] LNet: 45071:0:(api-ni.c:337:recovery_interval_set()) Skipped 1 previous similar message [ 315.718737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 315.734369] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 315.749775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 315.947602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_no_route [ 315.965676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_no_route [ 315.985383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 316.138371] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 316.156176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 316.171868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 316.418091] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_timeout [ 316.447950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_timeout [ 316.472234] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 316.666831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 316.687173] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 316.703201] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 316.922592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_error [ 316.943690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_error [ 316.966082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 316.982460] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 316.989178] LNet: Skipped 2 previous similar messages [ 317.134148] LNet: 45326:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 317.138424] LNet: 45326:0:(api-ni.c:337:recovery_interval_set()) Skipped 2 previous similar messages [ 317.181436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 317.204249] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 317.225904] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 317.349279] LNet: 45362:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 317.353410] LNetError: 45362:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 317.368954] LNet: Removed LNI 192.168.201.34@tcp [ 317.580861] Key type .llcrypt unregistered [ 317.582626] Key type ._llcrypt unregistered [ 324.333441] Key type ._llcrypt registered [ 324.335233] Key type .llcrypt registered [ 324.952722] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 16:59:46 (1755637186) [ 325.326610] Key type .llcrypt unregistered [ 325.328450] Key type ._llcrypt unregistered [ 327.528872] Key type ._llcrypt registered [ 327.530109] Key type .llcrypt registered [ 327.602190] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 327.609080] alg: No test for adler32 (adler32-zlib) [ 328.424404] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 328.437301] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 331.638811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 334.917191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 334.931188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 334.940048] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 335.357703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_interrupt [ 335.371034] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp1 -m GET -r 1 -e local_interrupt [ 335.386430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_interrupt [ 335.402182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp1 -m GET -r 1 -e local_interrupt [ 335.419850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp -m GET -r 1 -e local_interrupt [ 335.437260] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp1 -m GET -r 1 -e local_interrupt [ 335.455446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp -m GET -r 1 -e local_interrupt [ 335.469993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp1 -m GET -r 1 -e local_interrupt [ 335.486377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 335.495835] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 335.499532] LNet: 1 local NIs in recovery (showing 1): 192.168.201.34@tcp [ 335.597586] LNet: 46796:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 335.623431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 335.638697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 335.652020] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 335.801147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_dropped [ 335.815191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp1 -m GET -r 1 -e local_dropped [ 335.831080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_dropped [ 335.846266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp1 -m GET -r 1 -e local_dropped [ 335.861674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp -m GET -r 1 -e local_dropped [ 335.876700] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp1 -m GET -r 1 -e local_dropped [ 335.891628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp -m GET -r 1 -e local_dropped [ 335.907347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp1 -m GET -r 1 -e local_dropped [ 335.924317] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 336.046928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 336.060472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 336.073900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 336.229187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_aborted [ 336.242326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp1 -m GET -r 1 -e local_aborted [ 336.257463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_aborted [ 336.270434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp1 -m GET -r 1 -e local_aborted [ 336.283347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp -m GET -r 1 -e local_aborted [ 336.297304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp1 -m GET -r 1 -e local_aborted [ 336.309445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp -m GET -r 1 -e local_aborted [ 336.320368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp1 -m GET -r 1 -e local_aborted [ 336.331710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 336.342130] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 336.345578] LNet: Skipped 5 previous similar messages [ 336.422653] LNet: 46990:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 336.425506] LNet: 46990:0:(api-ni.c:337:recovery_interval_set()) Skipped 1 previous similar message [ 336.449286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 336.461164] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 336.472294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 336.613236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_no_route [ 336.625797] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp1 -m GET -r 1 -e local_no_route [ 336.637861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_no_route [ 336.650326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp1 -m GET -r 1 -e local_no_route [ 336.665168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp -m GET -r 1 -e local_no_route [ 336.679967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp1 -m GET -r 1 -e local_no_route [ 336.694747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp -m GET -r 1 -e local_no_route [ 336.710115] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp1 -m GET -r 1 -e local_no_route [ 336.723927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 336.843208] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 336.857298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 336.870784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 337.025550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_timeout [ 337.042649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp1 -m GET -r 1 -e local_timeout [ 337.059190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_timeout [ 337.076456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp1 -m GET -r 1 -e local_timeout [ 337.093424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp -m GET -r 1 -e local_timeout [ 337.109665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp1 -m GET -r 1 -e local_timeout [ 337.125228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp -m GET -r 1 -e local_timeout [ 337.141141] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp1 -m GET -r 1 -e local_timeout [ 337.159583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 337.287644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 337.301202] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 337.314658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 337.485308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e local_error [ 337.500901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp1 -m GET -r 1 -e local_error [ 337.515743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e local_error [ 337.530933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp1 -m GET -r 1 -e local_error [ 337.547774] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp -m GET -r 1 -e local_error [ 337.563839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp1 -m GET -r 1 -e local_error [ 337.580134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp -m GET -r 1 -e local_error [ 337.595176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp1 -m GET -r 1 -e local_error [ 337.609860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 337.619914] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 337.623720] LNet: Skipped 9 previous similar messages [ 337.709718] LNet: 47281:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 337.713033] LNet: 47281:0:(api-ni.c:337:recovery_interval_set()) Skipped 2 previous similar messages [ 337.737963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 337.750653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 337.764954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 338.880668] LNet: 47351:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 338.884417] LNetError: 47351:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 338.887907] LNet: Removed LNI 192.168.201.34@tcp [ 338.897333] LNet: Removed LNI 192.168.201.34@tcp1 [ 339.049622] Key type .llcrypt unregistered [ 339.051274] Key type ._llcrypt unregistered [ 344.285354] Key type ._llcrypt registered [ 344.286689] Key type .llcrypt registered [ 344.777585] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 17:00:06 (1755637206) [ 345.055274] Key type .llcrypt unregistered [ 345.057039] Key type ._llcrypt unregistered [ 346.954476] Key type ._llcrypt registered [ 346.956226] Key type .llcrypt registered [ 347.045300] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 347.055309] alg: No test for adler32 (adler32-zlib) [ 347.860351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 347.872185] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 350.906729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 351.145674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e remote_dropped [ 351.164828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e remote_dropped [ 351.184786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 351.195945] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 351.300588] LNet: 48573:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 351.332531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 351.349359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 351.365228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 351.547055] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e remote_error [ 351.562457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e remote_error [ 351.579330] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 351.723480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 351.738947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 351.755447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 351.953517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e remote_timeout [ 351.971271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e remote_timeout [ 351.988297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 351.997411] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 352.000866] LNet: Skipped 1 previous similar message [ 352.092201] LNet: 48743:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 352.095612] LNet: 48743:0:(api-ni.c:337:recovery_interval_set()) Skipped 1 previous similar message [ 352.125728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 352.140440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 352.156147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 352.258886] LNet: 48779:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 352.261781] LNetError: 48779:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 352.268425] LNet: Removed LNI 192.168.201.34@tcp [ 352.439609] Key type .llcrypt unregistered [ 352.441588] Key type ._llcrypt unregistered [ 357.718331] Key type ._llcrypt registered [ 357.719381] Key type .llcrypt registered [ 358.180192] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 17:00:20 (1755637220) [ 358.457269] Key type .llcrypt unregistered [ 358.458467] Key type ._llcrypt unregistered [ 360.246367] Key type ._llcrypt registered [ 360.247963] Key type .llcrypt registered [ 360.304641] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 360.310560] alg: No test for adler32 (adler32-zlib) [ 361.109908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 361.121768] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 363.954848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 366.658573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 366.671658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 366.679677] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 367.120520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e remote_dropped [ 367.135584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp1 -m GET -r 1 -e remote_dropped [ 367.151368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e remote_dropped [ 367.166585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp1 -m GET -r 1 -e remote_dropped [ 367.184425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp -m GET -r 1 -e remote_dropped [ 367.202152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp1 -m GET -r 1 -e remote_dropped [ 367.217637] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp -m GET -r 1 -e remote_dropped [ 367.231692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp1 -m GET -r 1 -e remote_dropped [ 367.248329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 367.256556] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 367.259726] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.134@tcp [ 367.344960] LNet: 50212:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 367.370121] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 367.386726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 367.399729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 367.413613] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 367.591402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e remote_error [ 367.605805] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp1 -m GET -r 1 -e remote_error [ 367.621593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e remote_error [ 367.639189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp1 -m GET -r 1 -e remote_error [ 367.656968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp -m GET -r 1 -e remote_error [ 367.675934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp1 -m GET -r 1 -e remote_error [ 367.692663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp -m GET -r 1 -e remote_error [ 367.709097] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp1 -m GET -r 1 -e remote_error [ 367.725238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 367.846941] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 367.862258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 367.878073] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 367.893057] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 368.055718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e remote_timeout [ 368.070057] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp1 -m GET -r 1 -e remote_timeout [ 368.083867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e remote_timeout [ 368.099467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp1 -m GET -r 1 -e remote_timeout [ 368.114452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp -m GET -r 1 -e remote_timeout [ 368.129977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp1 -m GET -r 1 -e remote_timeout [ 368.147727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp -m GET -r 1 -e remote_timeout [ 368.161762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp1 -m GET -r 1 -e remote_timeout [ 368.177313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.134@tcp [ 368.186353] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 368.188779] LNet: Skipped 3 previous similar messages [ 368.278127] LNet: 50410:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 368.281852] LNet: 50410:0:(api-ni.c:337:recovery_interval_set()) Skipped 1 previous similar message [ 368.308324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 368.322762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 368.335085] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 368.347869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 370.483539] LNet: 50483:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 370.486131] LNetError: 50483:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 370.489453] LNet: Removed LNI 192.168.201.34@tcp [ 370.494310] LNet: Removed LNI 192.168.201.34@tcp1 [ 370.646544] Key type .llcrypt unregistered [ 370.648063] Key type ._llcrypt unregistered [ 375.581545] Key type ._llcrypt registered [ 375.582976] Key type .llcrypt registered [ 376.054764] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 17:00:37 (1755637237) [ 376.221649] Key type .llcrypt unregistered [ 376.223521] Key type ._llcrypt unregistered [ 378.213064] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 378.266926] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 378.269871] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 378.273038] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 378.405888] Key type ._llcrypt registered [ 378.407170] Key type .llcrypt registered [ 378.469488] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 378.475449] alg: No test for adler32 (adler32-zlib) [ 379.286741] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 379.304117] LNet: 51547:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 379.308258] LNetError: 51547:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 379.317366] LNet: Removed LNI 192.168.201.34@tcp [ 379.583559] Key type .llcrypt unregistered [ 379.584805] Key type ._llcrypt unregistered [ 381.602278] Key type ._llcrypt registered [ 381.603870] Key type .llcrypt registered [ 381.671559] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 381.678361] alg: No test for adler32 (adler32-zlib) [ 382.487576] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 382.489389] LNet: Added LNI 10.1.2.3@tcp1 [8/256/0/180] [ 382.506029] LNet: 51816:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 382.509245] LNetError: 51816:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 382.511892] LNet: Removed LNI 192.168.201.34@tcp [ 382.517535] LNet: Removed LNI 10.1.2.3@tcp1 [ 382.773438] Key type .llcrypt unregistered [ 382.774669] Key type ._llcrypt unregistered [ 384.765476] Key type ._llcrypt registered [ 384.766760] Key type .llcrypt registered [ 384.826940] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 384.832107] alg: No test for adler32 (adler32-zlib) [ 385.647243] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 385.649478] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 385.665588] LNet: 52085:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 385.668674] LNetError: 52085:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 385.672442] LNet: Removed LNI 192.168.201.34@tcp [ 385.681343] LNet: Removed LNI 10.1.2.3@tcp [ 385.936554] Key type .llcrypt unregistered [ 385.938114] Key type ._llcrypt unregistered [ 387.956135] Key type ._llcrypt registered [ 387.957706] Key type .llcrypt registered [ 388.027170] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 388.034671] alg: No test for adler32 (adler32-zlib) [ 388.838356] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 388.840397] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 388.854473] LNet: 52354:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 388.857678] LNetError: 52354:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 388.860473] LNet: Removed LNI 192.168.201.34@tcp [ 388.869299] LNet: Removed LNI 10.1.2.3@tcp [ 389.115349] Key type .llcrypt unregistered [ 389.116912] Key type ._llcrypt unregistered [ 390.978877] Key type ._llcrypt registered [ 390.980288] Key type .llcrypt registered [ 391.039044] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 391.045356] alg: No test for adler32 (adler32-zlib) [ 391.847828] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 391.849867] LNetError: 52614:0:(config.c:1557:lnet_inet_select()) ksocklnd: failed to find UP interface test1pg [ 391.853208] LNetError: Error -100 starting up LNI tcp [ 391.866232] LNet: Removed LNI 192.168.201.34@tcp [ 392.099302] Key type .llcrypt unregistered [ 392.100587] Key type ._llcrypt unregistered [ 395.567377] Key type ._llcrypt registered [ 395.568676] Key type .llcrypt registered [ 396.019522] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 17:00:57 (1755637257) [ 396.307272] Key type .llcrypt unregistered [ 396.308894] Key type ._llcrypt unregistered [ 398.193131] Key type ._llcrypt registered [ 398.194659] Key type .llcrypt registered [ 398.264948] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 398.271639] alg: No test for adler32 (adler32-zlib) [ 399.076976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 399.088122] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 401.952943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 402.164428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e network_timeout [ 402.178767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e network_timeout [ 402.195168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 402.202831] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 402.286803] LNet: 53854:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 402.314188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 402.328415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 402.419458] LNet: 53889:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 402.423557] LNetError: 53889:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 402.432428] LNet: Removed LNI 192.168.201.34@tcp [ 402.599472] Key type .llcrypt unregistered [ 402.600755] Key type ._llcrypt unregistered [ 408.217913] Key type ._llcrypt registered [ 408.219302] Key type .llcrypt registered [ 408.286173] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 408.292659] alg: No test for adler32 (adler32-zlib) [ 409.096913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 409.108121] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 411.938341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 414.528223] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 414.542101] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 414.552772] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 415.062967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp -m GET -r 1 -e network_timeout [ 415.080675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp -d 192.168.201.134@tcp1 -m GET -r 1 -e network_timeout [ 415.097734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp -m GET -r 1 -e network_timeout [ 415.116837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp -d 192.168.201.34@tcp1 -m GET -r 1 -e network_timeout [ 415.136150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp -m GET -r 1 -e network_timeout [ 415.153502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.34@tcp1 -d 192.168.201.134@tcp1 -m GET -r 1 -e network_timeout [ 415.173680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp -m GET -r 1 -e network_timeout [ 415.193342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp1 -m GET -r 1 -e network_timeout [ 415.210586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 415.221493] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 415.265433] LNet: 1 local NIs in recovery (showing 1): 192.168.201.34@tcp [ 415.268171] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.134@tcp [ 415.334896] LNet: 54748:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 415.365911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 415.380966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 417.497181] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 417.600043] LNet: 54817:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 417.603468] LNetError: 54817:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 417.607523] LNet: Removed LNI 192.168.201.34@tcp [ 417.615398] LNet: Removed LNI 192.168.201.34@tcp1 [ 417.794670] Key type .llcrypt unregistered [ 417.796305] Key type ._llcrypt unregistered [ 423.318179] Key type ._llcrypt registered [ 423.319310] Key type .llcrypt registered [ 423.832744] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 17:01:25 (1755637285) [ 423.971142] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 423.978604] alg: No test for adler32 (adler32-zlib) [ 424.779860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 424.800303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 424.811216] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 425.079397] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 425.088918] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 425.402162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.34@tcp [ 425.427585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 425.448727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 425.465126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 425.480321] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 425.497140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 425.512171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.34@tcp [ 425.520693] LNetError: 55905:0:(api-ni.c:4220:lnet_ni_set_healthv()) manually adding local NI 192.168.201.34@tcp to recovery [ 425.824304] LNet: 1 local NIs in recovery (showing 1): 192.168.201.34@tcp [ 425.824346] LNet: There was an unexpected network error while writing to 192.168.201.34: rc = -22 [ 427.872829] LNet: There was an unexpected network error while writing to 192.168.201.34: rc = -22 [ 428.679927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 431.969392] LNet: There was an unexpected network error while writing to 192.168.201.34: rc = -22 [ 432.890359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 432.904192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 432.927834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 432.938819] LNet: 56023:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 432.942138] LNetError: 56023:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 432.945515] LNet: Removed LNI 192.168.201.34@tcp [ 432.958295] LNet: Removed LNI 192.168.201.34@tcp1 [ 432.966656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 432.991592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 433.004149] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 433.281137] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 433.291408] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 433.610747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.34@tcp [ 433.625439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 433.650407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 433.673320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 433.689727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 433.709751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 433.726801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 433.742511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.34@tcp [ 433.750162] LNetError: 56289:0:(api-ni.c:4220:lnet_ni_set_healthv()) manually adding local NI 192.168.201.34@tcp to recovery [ 434.016325] LNet: There was an unexpected network error while writing to 192.168.201.34: rc = -22 [ 436.911370] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 440.160388] LNet: There was an unexpected network error while writing to 192.168.201.34: rc = -22 [ 440.163936] LNet: Skipped 1 previous similar message [ 445.252262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 445.266024] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 445.280459] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 447.609731] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 17:01:49 (1755637309) [ 447.646507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 447.656855] LNet: 57013:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 447.659967] LNetError: 57013:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 447.663416] LNetError: 57013:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 447.667081] LNet: Removed LNI 192.168.201.34@tcp [ 447.673745] LNet: Removed LNI 192.168.201.34@tcp1 [ 447.683479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 447.705402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 447.717425] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 447.986869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 447.996648] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 448.296958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.34@tcp [ 448.324612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 448.341723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 448.358689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 448.374951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 448.388923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 448.402418] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.34@tcp --health 0 [ 448.439897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 448.736165] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.34@tcp [ 449.760365] LNet: There was an unexpected network error while writing to 192.168.201.34: rc = -22 [ 449.764170] LNet: Skipped 1 previous similar message [ 463.907648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 463.958759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 463.972766] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 463.985310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.34@tcp --health 500 [ 464.022932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 470.237292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 470.263538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 470.273508] LNet: 57509:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 470.276385] LNetError: 57509:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 470.279675] LNetError: 57509:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 470.283095] LNet: Removed LNI 192.168.201.34@tcp [ 470.288632] LNet: Removed LNI 192.168.201.34@tcp1 [ 470.297718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 470.322346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 470.333445] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 470.605445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 470.616772] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 470.958270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.34@tcp [ 470.976907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 471.004249] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 471.020380] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 471.036715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 471.053850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 471.070678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 471.086730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.34@tcp --health 0 [ 472.352349] LNet: There was an unexpected network error while writing to 192.168.201.34: rc = -22 [ 472.356015] LNet: Skipped 3 previous similar messages [ 473.251323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 483.669638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 483.683922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 483.700710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 486.233605] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 17:02:28 (1755637348) [ 486.407293] LNet: 58570:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 486.410690] LNetError: 58570:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 486.413813] LNetError: 58570:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 486.416716] LNet: Removed LNI 192.168.201.34@tcp [ 486.424481] LNet: Removed LNI 192.168.201.34@tcp1 [ 486.673715] Key type .llcrypt unregistered [ 486.674957] Key type ._llcrypt unregistered [ 488.621430] Key type ._llcrypt registered [ 488.622900] Key type .llcrypt registered [ 488.692786] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 488.700262] alg: No test for adler32 (adler32-zlib) [ 489.513868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 489.527888] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 493.058351] Lustre: DEBUG MARKER: Initial discovery [ 493.079784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.134@tcp [ 493.975301] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag [ 494.005902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 -e local_error [ 494.027320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.134@tcp [ 494.039176] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 494.052138] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 494.068548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.201.134@tcp -r 1 -m GET -l 3 [ 494.085881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.201.134@tcp -r 1 -m GET -e local_error [ 494.103038] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.201.134@tcp -r 1 -m PUT -l 6 [ 494.618683] Lustre: DEBUG MARKER: Force oleg134-client.virtnet to discover 192.168.201.134@tcp (in background) [ 494.644216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.134@tcp [ 495.162829] Lustre: DEBUG MARKER: Force oleg134-server to discover 192.168.201.34@tcp [ 496.051086] Lustre: DEBUG MARKER: Wait for 59319 [ 497.696406] LNet: There was an unexpected network error while writing to 192.168.201.134: rc = -22 [ 498.231587] Lustre: DEBUG MARKER: Finished wait on 59319 [ 498.250566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay del -a [ 498.272718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 498.393716] LNet: 59774:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 498.397514] LNetError: 59774:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 498.406368] LNet: Removed LNI 192.168.201.34@tcp [ 498.569940] Key type .llcrypt unregistered [ 498.571630] Key type ._llcrypt unregistered [ 504.210021] Key type ._llcrypt registered [ 504.211638] Key type .llcrypt registered [ 504.679517] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 17:02:46 (1755637366) [ 504.863736] Key type .llcrypt unregistered [ 504.865320] Key type ._llcrypt unregistered [ 506.889274] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 506.937374] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 506.941704] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 506.943993] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 507.067556] Key type ._llcrypt registered [ 507.069177] Key type .llcrypt registered [ 507.142329] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 507.148213] alg: No test for adler32 (adler32-zlib) [ 507.954886] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 507.978323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 507.989149] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 508.257051] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 508.267094] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 511.281219] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 17:02:53 (1755637373) [ 511.312475] LNet: 61805:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 511.316907] LNetError: 61805:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 511.320552] LNet: Removed LNI 192.168.201.34@tcp [ 511.328617] LNet: Removed LNI 10.1.2.3@tcp [ 511.573431] Key type .llcrypt unregistered [ 511.575116] Key type ._llcrypt unregistered [ 513.687580] Key type ._llcrypt registered [ 513.688847] Key type .llcrypt registered [ 513.757459] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 513.763968] alg: No test for adler32 (adler32-zlib) [ 514.569074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 514.593803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 514.606495] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 514.861614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 514.871244] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 515.222464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 10.1.2.3@tcp 192.168.201.34@tcp [ 516.256150] LNetError: Unexpected error -22 connecting to 192.168.201.34@tcp at host 192.168.201.34:988 [ 517.488787] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 17:02:59 (1755637379) [ 517.523071] LNet: 62919:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 517.526579] LNetError: 62919:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 517.529297] LNet: Removed LNI 192.168.201.34@tcp [ 517.534704] LNet: Removed LNI 10.1.2.3@tcp [ 517.802570] Key type .llcrypt unregistered [ 517.804882] Key type ._llcrypt unregistered [ 519.801216] Key type ._llcrypt registered [ 519.802707] Key type .llcrypt registered [ 519.885312] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 519.892153] alg: No test for adler32 (adler32-zlib) [ 520.700947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 520.725064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 520.736048] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 520.996361] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 521.005488] LNet: Added LNI 192.168.201.34@tcp2 [8/256/0/180] [ 521.306676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 192.168.201.34@tcp1 --nid 192.168.201.34@tcp2 [ 522.816890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.34@tcp1 192.168.201.34@tcp2 [ 522.826204] LNetError: 63750:0:(lib-move.c:2237:lnet_handle_find_routed_path()) Unable to send message from 192.168.201.34@tcp1 to 192.168.201.34@tcp2 - Route table may be misconfigured [ 522.839729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.34@tcp2 192.168.201.34@tcp1 [ 525.068793] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 17:03:06 (1755637386) [ 525.104907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 525.118967] LNet: 64317:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 525.122794] LNetError: 64317:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 525.125975] LNet: Removed LNI 192.168.201.34@tcp1 [ 526.182605] LNet: Removed LNI 192.168.201.34@tcp2 [ 526.193788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 526.217885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 526.229774] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 526.501911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 526.511507] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 526.841710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.34@tcp [ 526.860803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.34@tcp 192.168.201.34@tcp [ 526.878818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.34@tcp1 192.168.201.34@tcp1 [ 526.897679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.34@tcp -d 192.168.201.34@tcp -e network_timeout [ 526.916513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.201.34@tcp -d 192.168.201.34@tcp1 -e network_timeout [ 526.935755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp -e network_timeout [ 526.954841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.34@tcp1 -d 192.168.201.34@tcp1 -e network_timeout [ 526.971052] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 526.980547] LNet: There was an unexpected network error while writing to 192.168.201.34: rc = -22 [ 526.989597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 527.026199] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 527.065763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 527.265353] LNet: 1 local NIs in recovery (showing 1): 192.168.201.34@tcp [ 529.288212] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 17:03:11 (1755637391) [ 529.325884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 529.337165] LNet: 65175:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 529.340527] LNetError: 65175:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 529.344078] LNetError: 65175:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 529.348337] LNet: Removed LNI 192.168.201.34@tcp [ 530.406547] LNet: Removed LNI 192.168.201.34@tcp1 [ 530.416253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 530.447963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 0@lo [ 530.539417] LNet: 65215:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 530.691656] Key type .llcrypt unregistered [ 530.693410] Key type ._llcrypt unregistered [ 534.380553] Key type ._llcrypt registered [ 534.382063] Key type .llcrypt registered [ 534.890272] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 17:03:16 (1755637396) [ 535.017199] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 535.024244] alg: No test for adler32 (adler32-zlib) [ 535.827673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 536.398824] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces [ 537.001967] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 17:03:18 (1755637398) [ 537.032913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 537.045825] LNet: 66302:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 537.057795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 537.085515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 537.095761] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 537.343939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 537.352658] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 537.668664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 537.686245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp1 [ 537.702913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.34@tcp1 [ 539.815995] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 17:03:21 (1755637401) [ 540.436965] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_220 Need at least 2 remote nodes found oleg134-server [ 541.012871] Lustre: DEBUG MARKER: == sanity-lnet test 221: Add routes w/DD disabled - check aliveness ========================================================== 17:03:22 (1755637402) [ 541.634039] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_221 Need at least 2 remote nodes found oleg134-server [ 542.240198] Lustre: DEBUG MARKER: == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 17:03:24 (1755637404) [ 542.872050] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_222 Need at least 2 remote nodes found oleg134-server [ 543.432946] Lustre: DEBUG MARKER: == sanity-lnet test 223: Check avoid_asym_router_failure=1 w/DD disabled ========================================================== 17:03:25 (1755637405) [ 544.009326] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_223 Need at least 2 remote nodes found oleg134-server [ 544.521273] Lustre: DEBUG MARKER: == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 17:03:26 (1755637406) [ 545.135844] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_224 Need at least 2 remote nodes found oleg134-server [ 545.726040] Lustre: DEBUG MARKER: == sanity-lnet test 225: Check avoid_asym_router_failure=0 w/DD disabled ========================================================== 17:03:27 (1755637407) [ 546.354635] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_225 Need at least 2 remote nodes found oleg134-server [ 546.940907] Lustre: DEBUG MARKER: == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 17:03:28 (1755637408) [ 547.584555] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_226 Need at least 3 remote nodes found oleg134-server [ 548.180199] Lustre: DEBUG MARKER: == sanity-lnet test 227: Check router peer health w/DD disabled ========================================================== 17:03:30 (1755637410) [ 548.842295] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_227 Need at least 3 remote nodes found oleg134-server [ 549.455570] Lustre: DEBUG MARKER: == sanity-lnet test 230: Test setting conns-per-peer ===== 17:03:31 (1755637411) [ 549.511542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 549.524527] LNet: 69836:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 549.527916] LNetError: 69836:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 549.530779] LNet: Removed LNI 192.168.201.34@tcp [ 550.566802] LNet: Removed LNI 192.168.201.34@tcp1 [ 550.575592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 550.599641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 550.609790] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 550.864408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 4 [ 550.897951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 550.969161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 550.977637] LNet: 69980:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 550.980782] LNetError: 69980:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 550.983717] LNetError: 69980:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 550.996660] LNet: Removed LNI 192.168.201.34@tcp [ 551.005903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 551.029762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 5 [ 551.041644] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 551.361481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 551.429580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 552.486350] LNet: Removed LNI 192.168.201.34@tcp [ 552.494949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 552.520627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 552.532075] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 552.833977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 6 [ 552.869898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 552.941636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 552.951672] LNetError: 70266:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 552.955372] LNetError: 70266:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 552.969595] LNet: Removed LNI 192.168.201.34@tcp [ 552.979959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 553.003869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 7 [ 553.014913] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 553.336902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 553.416169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 553.427855] LNet: 70408:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 553.430592] LNet: 70408:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 554.470705] LNet: Removed LNI 192.168.201.34@tcp [ 554.479614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 554.504674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 554.515385] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 554.795044] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 8 [ 554.834640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 554.914641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 555.942442] LNet: Removed LNI 192.168.201.34@tcp [ 555.951903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 555.978072] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 9 [ 555.990982] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 556.308610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 556.395192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 556.405547] LNetError: 70695:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 556.408311] LNetError: 70695:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 556.421594] LNet: Removed LNI 192.168.201.34@tcp [ 556.431620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 556.457833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 556.470538] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 556.768412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 10 [ 556.803901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 556.894826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 557.926399] LNet: Removed LNI 192.168.201.34@tcp [ 557.936463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 557.962520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 11 [ 557.974592] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 558.273785] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 558.358902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 558.367116] LNet: 70981:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 558.369230] LNet: 70981:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 559.398475] LNet: Removed LNI 192.168.201.34@tcp [ 559.406436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 559.428093] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 559.438345] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 559.676983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 12 [ 559.711386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 559.798777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 560.870318] LNet: Removed LNI 192.168.201.34@tcp [ 560.879362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 560.902200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 13 [ 560.912746] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 561.196777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 561.283266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 561.294110] LNetError: 71268:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 561.296406] LNetError: 71268:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 561.303741] LNet: Removed LNI 192.168.201.34@tcp [ 561.312517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 561.333174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 561.343481] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 561.600251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 14 [ 561.633344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 561.716630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 562.790605] LNet: Removed LNI 192.168.201.34@tcp [ 562.799444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 562.824548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 15 [ 562.837427] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 563.123580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 563.218273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 564.262379] LNet: Removed LNI 192.168.201.34@tcp [ 564.271220] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 564.295874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 564.307145] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 564.579131] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 128 [ 564.602434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 564.618326] LNet: Removed LNI 192.168.201.34@tcp [ 564.627058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 564.653475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 564.667255] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 564.951955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer -1 [ 567.383300] Lustre: DEBUG MARKER: == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 17:03:49 (1755637429) [ 567.417192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 567.427485] LNet: 72379:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 567.430782] LNet: 72379:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 567.442620] LNet: Removed LNI 192.168.201.34@tcp [ 567.453254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 567.475730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 [ 567.485930] LNet: Added LNI 192.168.201.34@tcp231 [8/256/0/180] [ 567.946231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 567.962666] LNet: Removed LNI 192.168.201.34@tcp231 [ 567.971453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 568.005565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 568.023120] LNet: Added LNI 192.168.201.34@tcp231 [8/256/0/180] [ 568.028879] LNet: 72527:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 568.190646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 568.207504] LNet: Removed LNI 192.168.201.34@tcp231 [ 568.216076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=-1 [ 568.227367] LNet: Added LNI 192.168.201.34@tcp231 [8/256/0/180] [ 568.654326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 568.669366] LNet: Removed LNI 192.168.201.34@tcp231 [ 568.677906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 568.708826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 568.721857] LNet: Added LNI 192.168.201.34@tcp231 [8/256/0/0] [ 568.724938] LNet: 72676:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 568.887973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 568.906517] LNet: Removed LNI 192.168.201.34@tcp231 [ 568.914675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=0 [ 568.925934] LNet: Added LNI 192.168.201.34@tcp231 [8/256/0/0] [ 569.350767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 569.360786] LNetError: 72816:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 569.363551] LNetError: 72816:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Skipped 6 previous similar messages [ 569.373474] LNet: Removed LNI 192.168.201.34@tcp231 [ 569.382532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 569.411266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 569.422096] LNet: Added LNI 192.168.201.34@tcp231 [8/256/0/60] [ 569.581895] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 569.601508] LNet: Removed LNI 192.168.201.34@tcp231 [ 569.608996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=60 [ 569.622343] LNet: Added LNI 192.168.201.34@tcp231 [8/256/0/60] [ 570.057460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 570.075603] LNet: Removed LNI 192.168.201.34@tcp231 [ 570.085194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 570.114645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 570.126304] LNet: Added LNI 192.168.201.34@tcp231 [8/256/0/180] [ 570.129174] LNet: 72972:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 570.132458] LNet: 72972:0:(api-ni.c:337:recovery_interval_set()) Skipped 1 previous similar message [ 570.283645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 570.301420] LNet: Removed LNI 192.168.201.34@tcp231 [ 570.308800] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=180 [ 570.321777] LNet: Added LNI 192.168.201.34@tcp231 [8/256/0/180] [ 572.782503] Lustre: DEBUG MARKER: == sanity-lnet test 232: Test setting ToS value ========== 17:03:54 (1755637434) [ 572.814802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 572.831815] LNet: Removed LNI 192.168.201.34@tcp231 [ 572.844349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 572.877912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 [ 572.888795] LNet: Added LNI 192.168.201.34@tcp232 [8/256/0/180] [ 573.137494] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 573.153950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 573.169816] LNet: Removed LNI 192.168.201.34@tcp232 [ 573.176771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos -1 [ 573.187485] LNet: Added LNI 192.168.201.34@tcp232 [8/256/0/180] [ 573.443379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 573.458382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 573.476565] LNet: Removed LNI 192.168.201.34@tcp232 [ 573.483756] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 104 [ 573.495117] LNet: Added LNI 192.168.201.34@tcp232 [8/256/0/180] [ 573.744178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 573.758687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 573.772451] LNet: Removed LNI 192.168.201.34@tcp232 [ 573.778778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 106 [ 573.789186] LNet: Added LNI 192.168.201.34@tcp232 [8/256/0/180] [ 574.034895] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 574.051697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 574.069397] LNet: Removed LNI 192.168.201.34@tcp232 [ 574.076731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 300 [ 576.179565] Lustre: DEBUG MARKER: == sanity-lnet test 241: Check setting LND timeout value via lnetctl updates tunables ========================================================== 17:03:58 (1755637438) [ 576.214350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 576.234727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 576.265276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 576.273842] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 576.518343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set -h [ 576.545733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 51 [ 576.564840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 576.593723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 0 [ 576.615304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 576.635812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 50 [ 578.703829] Lustre: DEBUG MARKER: == sanity-lnet test 250: test that linux routes are added ========================================================== 17:04:00 (1755637440) [ 578.733629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 578.754789] LNet: Removed LNI 192.168.201.34@tcp [ 578.764845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 578.790783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 578.799775] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 581.070981] Lustre: DEBUG MARKER: == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 17:04:02 (1755637442) [ 581.556040] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_251 Need kfi NETTYPE [ 582.132369] Lustre: DEBUG MARKER: == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 17:04:04 (1755637444) [ 582.289775] LNet: Removed LNI 192.168.201.34@tcp [ 582.533322] Key type .llcrypt unregistered [ 582.534887] Key type ._llcrypt unregistered [ 584.431440] Key type ._llcrypt registered [ 584.432870] Key type .llcrypt registered [ 584.505232] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 584.511697] alg: No test for adler32 (adler32-zlib) [ 585.319809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 585.331440] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 588.144767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 589.672883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --timeout 15 192.168.201.134@tcp [ 589.786349] LNet: 76913:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 589.790590] LNetError: 76913:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 589.799460] LNet: Removed LNI 192.168.201.34@tcp [ 589.967672] Key type .llcrypt unregistered [ 589.969735] Key type ._llcrypt unregistered [ 593.672733] Key type ._llcrypt registered [ 593.674328] Key type .llcrypt registered [ 594.185389] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253 [ 594.766707] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254 [ 595.357020] Lustre: DEBUG MARKER: == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 17:04:17 (1755637457) [ 595.481618] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 595.487668] alg: No test for adler32 (adler32-zlib) [ 596.294258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 596.329095] LNet: 77995:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 596.580467] Key type .llcrypt unregistered [ 596.581694] Key type ._llcrypt unregistered [ 598.586235] Key type ._llcrypt registered [ 598.587743] Key type .llcrypt registered [ 598.657606] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 598.665750] alg: No test for adler32 (adler32-zlib) [ 599.479655] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 599.482135] LNet: 78246:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 599.495236] LNetError: 78258:0:(router.c:400:lnet_router_discovery_ping_reply()) Peer 192.168.201.35@tcp is being used as a gateway but routing feature is not turned on [ 599.503528] LNetError: 78258:0:(lib-lnet.h:1327:lnet_set_route_aliveness()) route to o2ib through 192.168.201.35@tcp has gone from up to down [ 599.567873] LNetError: 78279:0:(router.c:334:lnet_is_route_alive()) peer 192.168.201.35@tcp is being used as a gateway but routing feature is not turned on [ 599.668720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 602.784169] LNetError: 78258:0:(lib-lnet.h:1327:lnet_set_route_aliveness()) route to o2ib through 192.168.201.39@tcp has gone from up to down [ 602.788391] LNetError: 78258:0:(lib-lnet.h:1327:lnet_set_route_aliveness()) Skipped 1 previous similar message [ 602.792166] LNet: 78304:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 602.795396] LNetError: 78304:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 602.808638] LNet: Removed LNI 192.168.201.34@tcp [ 602.818649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 602.827466] LNet: 78306:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 605.077306] Lustre: DEBUG MARKER: == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 17:04:26 (1755637466) [ 605.715406] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_256 Need at least 2 remote nodes found oleg134-server [ 606.267715] Lustre: DEBUG MARKER: == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 17:04:28 (1755637468) [ 606.864528] Lustre: Lustre: Build Version: 2.16.57_62_g5cfb5f1 [ 606.943676] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 606.945576] LNet: 79315:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 606.948843] LNet: 79315:0:(router.c:737:lnet_add_route()) Skipped 4 previous similar messages [ 606.955136] LNetError: 79330:0:(router.c:400:lnet_router_discovery_ping_reply()) Peer 192.168.201.35@tcp is being used as a gateway but routing feature is not turned on [ 606.962030] LNetError: 79330:0:(router.c:400:lnet_router_discovery_ping_reply()) Skipped 2 previous similar messages [ 606.966042] LNetError: 79330:0:(lib-lnet.h:1327:lnet_set_route_aliveness()) route to o2ib through 192.168.201.35@tcp has gone from up to down [ 606.969394] LNetError: 79330:0:(lib-lnet.h:1327:lnet_set_route_aliveness()) Skipped 2 previous similar messages [ 608.528131] Key type lgssc registered [ 608.936294] Lustre: Echo OBD driver; http://www.lustre.org/ [ 610.016348] LNetError: 79334:0:(lib-lnet.h:1327:lnet_set_route_aliveness()) route to o2ib through 192.168.201.38@tcp has gone from up to down [ 610.021968] LNetError: 79334:0:(lib-lnet.h:1327:lnet_set_route_aliveness()) Skipped 2 previous similar messages [ 616.573580] Key type lgssc unregistered [ 616.711559] LNet: 80355:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 616.714265] LNetError: 80355:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 616.724496] LNet: Removed LNI 192.168.201.34@tcp [ 616.965533] Key type .llcrypt unregistered [ 616.966824] Key type ._llcrypt unregistered [ 621.730683] Key type ._llcrypt registered [ 621.731874] Key type .llcrypt registered [ 622.183922] Lustre: DEBUG MARKER: == sanity-lnet test 280: Don't panic when request_module fails ========================================================== 17:04:44 (1755637484) [ 622.293335] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 622.298586] alg: No test for adler32 (adler32-zlib) [ 623.151238] LNetError: 81193:0:(api-ni.c:2619:lnet_load_lnd()) Can't load LND tcp, module ksocklnd, rc=256 [ 623.281735] Key type .llcrypt unregistered [ 623.283099] Key type ._llcrypt unregistered [ 625.020083] Key type ._llcrypt registered [ 625.021659] Key type .llcrypt registered [ 625.476949] Lustre: DEBUG MARKER: == sanity-lnet test 290: Check that lnet acceptor is using explicit address binding ========================================================== 17:04:47 (1755637487) [ 625.654438] Key type .llcrypt unregistered [ 625.655925] Key type ._llcrypt unregistered [ 627.447785] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 627.449493] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 627.528418] Key type ._llcrypt registered [ 627.530190] Key type .llcrypt registered [ 627.592074] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 627.597646] alg: No test for adler32 (adler32-zlib) [ 628.394165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 628.417177] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 632.191162] Lustre: DEBUG MARKER: == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 17:04:54 (1755637494) [ 632.276725] LNet: 82939:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 632.280506] LNetError: 82939:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface test1pg not found [ 632.289463] LNet: Removed LNI 10.1.2.3@tcp [ 632.540397] Key type .llcrypt unregistered [ 632.541934] Key type ._llcrypt unregistered [ 634.476218] Key type ._llcrypt registered [ 634.477781] Key type .llcrypt registered [ 634.554345] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 634.561152] alg: No test for adler32 (adler32-zlib) [ 637.669673] Lustre: DEBUG MARKER: == sanity-lnet test 301: Check for dynamic adds of same/wrong interface (memory leak) ========================================================== 17:04:59 (1755637499) [ 637.701251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 637.715944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 637.744763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 637.756121] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 638.008940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 638.023521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if ens2 [ 638.043563] LNet: Removed LNI 192.168.201.34@tcp [ 638.134880] LNet: 83911:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 638.294535] Key type .llcrypt unregistered [ 638.296108] Key type ._llcrypt unregistered [ 641.831647] Key type ._llcrypt registered [ 641.832625] Key type .llcrypt registered [ 642.290301] Lustre: DEBUG MARKER: == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 17:05:04 (1755637504) [ 642.406408] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 642.411946] alg: No test for adler32 (adler32-zlib) [ 643.217791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 643.242187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 643.253771] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 643.524213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 643.539603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug peer --nid 192.168.201.34@tcp [ 643.548857] LNet: Conn 0 [type, closing, crefcnt, srefcnt]: 2, 0, 1, 1 [ 643.550487] LNet: Conn 0 rx [scheduled, ready, state]: 0, 0, 1 [ 643.552252] LNet: Conn 0 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 643, 0, 0 [ 643.554422] LNet: Conn 0 sched [nconns, cpt]: 4, 0 [ 643.555804] LNet: Conn 1 [type, closing, crefcnt, srefcnt]: 3, 0, 1, 1 [ 643.557845] LNet: Conn 1 rx [scheduled, ready, state]: 0, 0, 1 [ 643.559271] LNet: Conn 1 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 643, 0, 0 [ 643.561787] LNet: Conn 1 sched [nconns, cpt]: 4, 0 [ 643.562929] LNet: Conn 2 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 643.564886] LNet: Conn 2 rx [scheduled, ready, state]: 0, 0, 1 [ 643.566965] LNet: Conn 2 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 643, 0, 693 [ 643.569193] LNet: Conn 2 sched [nconns, cpt]: 4, 0 [ 643.571096] LNet: Conn 3 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 643.572953] LNet: Conn 3 rx [scheduled, ready, state]: 0, 0, 1 [ 643.574942] LNet: Conn 3 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 643, 0, 693 [ 643.577056] LNet: Conn 3 sched [nconns, cpt]: 4, 0 [ 645.716943] Lustre: DEBUG MARKER: == sanity-lnet test 303: Check peer NI health after link down ========================================================== 17:05:07 (1755637507) [ 645.854417] LNet: 85445:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 645.857720] LNetError: 85445:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 645.866751] LNet: Removed LNI 192.168.201.34@tcp [ 646.096453] Key type .llcrypt unregistered [ 646.097703] Key type ._llcrypt unregistered [ 647.923350] Key type ._llcrypt registered [ 647.924623] Key type .llcrypt registered [ 647.989753] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 647.995895] alg: No test for adler32 (adler32-zlib) [ 648.800947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 648.811900] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 651.563302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 654.093282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 654.105218] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 654.113425] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 654.387873] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 654.437753] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 654.440140] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 654.442980] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 654.483974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp99 --if test1pg [ 654.493273] LNet: Added LNI 10.1.2.3@tcp99 [8/256/0/180] [ 657.842849] Lustre: DEBUG MARKER: == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 17:05:19 (1755637519) [ 657.885253] LNet: 86907:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 657.888110] LNetError: 86907:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 657.890659] LNet: Removed LNI 192.168.201.34@tcp [ 658.912156] LNet: Removed LNI 192.168.201.34@tcp1 [ 658.913643] LNetError: 86907:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface test1pg not found [ 658.915921] LNetError: 86907:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 658.925515] LNet: Removed LNI 10.1.2.3@tcp99 [ 659.148452] Key type .llcrypt unregistered [ 659.149922] Key type ._llcrypt unregistered [ 661.144181] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 661.147100] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 661.232717] Key type ._llcrypt registered [ 661.234338] Key type .llcrypt registered [ 661.299569] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 661.305451] alg: No test for adler32 (adler32-zlib) [ 662.112284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 662.135431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 662.144053] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 662.383515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 662.392796] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 663.138000] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.1.2.3@tcp --lock_prim [ 663.157252] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 663.175149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 10.1.2.3@tcp --state 1048577 [ 663.186033] LNetError: 87621:0:(peer.c:2209:lnet_del_peer_ni()) peer 10.1.2.3@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 663.200022] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 663.217206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.34@tcp [ 663.378437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 663.409384] LNet: 87641:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 663.413359] LNetError: 87641:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 663.417461] LNet: Removed LNI 192.168.201.34@tcp [ 663.427143] LNet: Removed LNI 10.1.2.3@tcp [ 663.691433] Key type .llcrypt unregistered [ 663.693122] Key type ._llcrypt unregistered [ 667.759238] Key type ._llcrypt registered [ 667.760842] Key type .llcrypt registered [ 668.259813] Lustre: DEBUG MARKER: == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 17:05:30 (1755637530) [ 668.408066] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 668.414305] alg: No test for adler32 (adler32-zlib) [ 669.226977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 669.250479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 669.261720] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 669.540895] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.34@tcp [ 669.576713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping oleg134-client@tcp [ 671.885330] Lustre: DEBUG MARKER: == sanity-lnet test 310: Set timeout and verify ========== 17:05:33 (1755637533) [ 671.934832] LNet: 89161:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 671.937906] LNetError: 89161:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 671.949641] LNet: Removed LNI 192.168.201.34@tcp [ 672.200501] Key type .llcrypt unregistered [ 672.202066] Key type ._llcrypt unregistered [ 674.398803] Key type ._llcrypt registered [ 674.400669] Key type .llcrypt registered [ 674.464227] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 674.471177] alg: No test for adler32 (adler32-zlib) [ 675.273878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure --all [ 675.285769] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 675.300090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 677.468682] Lustre: DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 17:05:39 (1755637539) [ 677.981223] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type [ 678.574293] Lustre: DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 17:05:40 (1755637540) [ 679.093465] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type [ 679.678375] Lustre: DEBUG MARKER: == sanity-lnet test 400: Check for udsp add/delete net rule without net num ========================================================== 17:05:41 (1755637541) [ 679.710996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 679.723409] LNet: 90574:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 679.727312] LNetError: 90574:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 679.737259] LNet: Removed LNI 192.168.201.34@tcp [ 679.747499] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 679.771625] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --priority 0 [ 679.787302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 680.048845] Key type .llcrypt unregistered [ 680.050425] Key type ._llcrypt unregistered [ 684.000171] Key type ._llcrypt registered [ 684.001955] Key type .llcrypt registered [ 684.504322] Lustre: DEBUG MARKER: == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 17:05:46 (1755637546) [ 684.627863] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 684.634392] alg: No test for adler32 (adler32-zlib) [ 685.443326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 685.466706] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 685.477083] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 685.727597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 685.748700] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.34@tcp [ 688.038717] Lustre: DEBUG MARKER: == sanity-lnet test 402: Destination net rule should not panic ========================================================== 17:05:49 (1755637549) [ 688.074713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 688.084204] LNet: 92121:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 688.087467] LNetError: 92121:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 688.096592] LNet: Removed LNI 192.168.201.34@tcp [ 688.110694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 688.136639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst kfi --priority 0 [ 688.150739] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 402@kfi [ 690.227311] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_410 skipping ALWAYS excluded test 410 [ 690.758672] Lustre: DEBUG MARKER: == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 17:05:52 (1755637552) [ 690.793661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 690.803671] LNet: 92841:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 690.815665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 690.889726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 690.898329] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 691.126140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 691.133256] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 691.429533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.34@tcp [ 691.447990] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d *@tcp -r 1 -l 1 -m PUT [ 691.471381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 192.168.201.34@tcp -m PUT -e local_timeout -r 1 [ 691.494579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 10.1.2.3@tcp -m PUT -e local_timeout -r 1 [ 691.514691] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 691.516888] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 691.589763] LNetError: 92870:0:(lib-move.c:4921:lnet_parse()) 192.168.201.34@tcp, src 192.168.201.34@tcp: Dropping PUT (error -108 looking up sender) [ 692.640470] LNet: 93146:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 692.642434] LNetError: 93146:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 692.644391] LNet: Removed LNI 192.168.201.34@tcp [ 692.649257] LNet: Removed LNI 10.1.2.3@tcp [ 692.789527] Key type .llcrypt unregistered [ 692.791259] Key type ._llcrypt unregistered [ 696.199348] Key type ._llcrypt registered [ 696.200767] Key type .llcrypt registered [ 696.666279] Lustre: DEBUG MARKER: == sanity-lnet test 501: Verify lnetctl peer set --health (SR) ========================================================== 17:05:58 (1755637558) [ 696.796717] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 696.802837] alg: No test for adler32 (adler32-zlib) [ 697.613784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 697.759536] LNet: 93994:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 698.010312] Key type .llcrypt unregistered [ 698.011571] Key type ._llcrypt unregistered [ 699.915169] Key type ._llcrypt registered [ 699.916412] Key type .llcrypt registered [ 699.983774] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 699.990918] alg: No test for adler32 (adler32-zlib) [ 700.795355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 700.806299] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 703.655626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 703.720733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 703.902666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.134@tcp [ 704.008760] LNet: 94418:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 704.040081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 704.054303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 704.245427] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 704.370883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 704.384901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 704.590976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.134@tcp [ 704.696786] LNet: 94604:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 704.699109] LNet: 94604:0:(api-ni.c:337:recovery_interval_set()) Skipped 1 previous similar message [ 704.723166] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 704.737334] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 704.945915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 705.069202] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 705.083134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 705.188651] LNet: 94738:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 705.190820] LNetError: 94738:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 705.199343] LNet: Removed LNI 192.168.201.34@tcp [ 705.349753] Key type .llcrypt unregistered [ 705.351134] Key type ._llcrypt unregistered [ 710.376832] Key type ._llcrypt registered [ 710.378068] Key type .llcrypt registered [ 710.822736] Lustre: DEBUG MARKER: == sanity-lnet test 502: Verify lnetctl peer set --health (MR) ========================================================== 17:06:12 (1755637572) [ 710.959967] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 710.965234] alg: No test for adler32 (adler32-zlib) [ 711.763967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 711.886803] LNet: 95621:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 712.107460] Key type .llcrypt unregistered [ 712.109365] Key type ._llcrypt unregistered [ 714.002820] Key type ._llcrypt registered [ 714.004332] Key type .llcrypt registered [ 714.073648] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 714.080194] alg: No test for adler32 (adler32-zlib) [ 714.884202] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 714.895072] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 717.593266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.134@tcp [ 720.116250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 720.128672] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 720.136430] LNet: Added LNI 192.168.201.34@tcp1 [8/256/0/180] [ 720.457672] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 720.635058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.134@tcp [ 720.722545] LNet: 96244:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 720.749481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 720.761815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 720.986127] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 721.056636] LNet: 2 peer NIs in recovery (showing 2): 192.168.201.134@tcp, 192.168.201.134@tcp1 [ 721.111151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 721.126069] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 721.321113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.134@tcp [ 721.408030] LNet: 96430:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 721.410658] LNet: 96430:0:(api-ni.c:337:recovery_interval_set()) Skipped 1 previous similar message [ 721.437431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 721.452192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 721.655718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 721.780833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 721.795730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 723.824563] LNet: 96599:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 723.827314] LNetError: 96599:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 723.829711] LNet: Removed LNI 192.168.201.34@tcp [ 723.843303] LNet: Removed LNI 192.168.201.34@tcp1 [ 724.008550] Key type .llcrypt unregistered [ 724.009871] Key type ._llcrypt unregistered [ 728.774206] Key type ._llcrypt registered [ 728.775250] Key type .llcrypt registered [ 729.194529] Lustre: DEBUG MARKER: == sanity-lnet test complete, duration 608 sec =========== 17:06:31 (1755637591) [ 729.414713] Key type .llcrypt unregistered [ 729.415780] Key type ._llcrypt unregistered [ 733.015112] Key type ._llcrypt registered [ 733.017143] Key type .llcrypt registered [ 733.152503] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 733.157548] alg: No test for adler32 (adler32-zlib) [ 733.999333] Lustre: Lustre: Build Version: 2.16.57_62_g5cfb5f1 [ 734.074674] LNet: Added LNI 192.168.201.34@tcp [8/256/0/180] [ 735.656117] Key type lgssc registered [ 736.025255] Lustre: Echo OBD driver; http://www.lustre.org/ [ 756.180930] Lustre: Mounted lustre-client [ 758.454540] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 765.447191] LustreError: 99786:0:(lov_obd.c:784:lov_cleanup()) lustre-clilov-ffff89fed8940800: lov tgt 0 not cleaned! deathrow=0, lovrc=1 [ 765.452651] LustreError: 99786:0:(obd_class.h:479:obd_check_dev()) Device 5 not setup [ 765.463089] Lustre: Unmounted lustre-client [ 790.538919] Key type lgssc unregistered [ 790.669945] LNet: 100264:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 790.672885] LNetError: 100264:0:(acceptor.c:254:lnet_acceptor_remove_socket()) Interface ens2 not found [ 790.681427] LNet: Removed LNI 192.168.201.34@tcp [ 790.971110] Key type .llcrypt unregistered [ 790.972408] Key type ._llcrypt unregistered