[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 402491574 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54d0-0x000f54df] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829700K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002341] x2apic enabled [ 0.003005] Switched APIC routing to physical x2apic. [ 0.004008] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008009] pid_max: default: 32768 minimum: 301 [ 0.009120] LSM: Security Framework initializing [ 0.010037] Yama: becoming mindful. [ 0.011025] SELinux: Initializing. [ 0.012064] *** VALIDATE selinux *** [ 0.019680] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.023782] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.024143] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026044] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027098] *** VALIDATE tmpfs *** [ 0.028427] *** VALIDATE proc *** [ 0.029203] *** VALIDATE cgroup *** [ 0.030006] *** VALIDATE cgroup2 *** [ 0.031232] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.033107] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.034005] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.035025] Spectre V2 : User space: Vulnerable [ 0.036006] Speculative Store Bypass: Vulnerable [ 0.038672] debug: unmapping init [mem 0xffffffffbc659000-0xffffffffbc660fff] [ 0.040970] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.041503] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.042014] ... version: 2 [ 0.043006] ... bit width: 48 [ 0.043830] ... generic registers: 4 [ 0.044010] ... value mask: 0000ffffffffffff [ 0.045009] ... max period: 00007fffffffffff [ 0.046010] ... fixed-purpose events: 3 [ 0.047009] ... event mask: 000000070000000f [ 0.048260] rcu: Hierarchical SRCU implementation. [ 0.050185] smp: Bringing up secondary CPUs ... [ 0.051542] x86: Booting SMP configuration: [ 0.052019] .... node #0, CPUs: #1 #2 #3 [ 0.055019] smp: Brought up 1 node, 4 CPUs [ 0.057011] smpboot: Max logical packages: 1 [ 0.058012] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.262167] node 0 deferred pages initialised in 203ms [ 0.265479] devtmpfs: initialized [ 0.266312] x86/mm: Memory block size: 128MB [ 0.269831] gcov: version magic: 0x41383552 [ 0.272429] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.275166] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.277384] pinctrl core: initialized pinctrl subsystem [ 0.279127] [ 0.279691] ************************************************************* [ 0.282014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.283008] ** ** [ 0.285012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.287008] ** ** [ 0.289016] ** This means that this kernel is built to expose internal ** [ 0.291013] ** IOMMU data structures, which may compromise security on ** [ 0.292011] ** your system. ** [ 0.294012] ** ** [ 0.296009] ** If you see this message and you are not debugging the ** [ 0.298008] ** kernel, report this immediately to your vendor! ** [ 0.300012] ** ** [ 0.302007] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.303009] ************************************************************* [ 0.305722] NET: Registered protocol family 16 [ 0.307457] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.309076] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.311069] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.315014] cpuidle: using governor menu [ 0.316525] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.318385] PCI: Using configuration type 1 for base access [ 0.320108] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.326137] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.329022] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.332090] cryptd: max_cpu_qlen set to 1000 [ 0.334231] ACPI: Added _OSI(Module Device) [ 0.335011] ACPI: Added _OSI(Processor Device) [ 0.336008] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.338009] ACPI: Added _OSI(Processor Aggregator Device) [ 0.341814] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.346588] ACPI: Interpreter enabled [ 0.347037] ACPI: PM: (supports S0 S3 S4 S5) [ 0.347986] ACPI: Using IOAPIC for interrupt routing [ 0.349074] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.352419] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.361090] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.363023] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.365013] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.367062] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.371198] acpiphp: Slot [2] registered [ 0.373079] acpiphp: Slot [5] registered [ 0.374087] acpiphp: Slot [6] registered [ 0.375064] acpiphp: Slot [3] registered [ 0.376063] acpiphp: Slot [4] registered [ 0.377006] acpiphp: Slot [7] registered [ 0.378058] acpiphp: Slot [8] registered [ 0.379059] acpiphp: Slot [9] registered [ 0.380025] acpiphp: Slot [10] registered [ 0.381062] acpiphp: Slot [11] registered [ 0.382071] acpiphp: Slot [12] registered [ 0.383051] acpiphp: Slot [13] registered [ 0.384066] acpiphp: Slot [14] registered [ 0.385019] acpiphp: Slot [15] registered [ 0.387064] acpiphp: Slot [16] registered [ 0.388045] acpiphp: Slot [17] registered [ 0.389066] acpiphp: Slot [18] registered [ 0.390088] acpiphp: Slot [19] registered [ 0.391059] acpiphp: Slot [20] registered [ 0.392055] acpiphp: Slot [21] registered [ 0.392942] acpiphp: Slot [22] registered [ 0.394063] acpiphp: Slot [23] registered [ 0.395091] acpiphp: Slot [24] registered [ 0.396075] acpiphp: Slot [25] registered [ 0.398093] acpiphp: Slot [26] registered [ 0.400075] acpiphp: Slot [27] registered [ 0.401072] acpiphp: Slot [28] registered [ 0.403078] acpiphp: Slot [29] registered [ 0.404068] acpiphp: Slot [30] registered [ 0.406099] acpiphp: Slot [31] registered [ 0.407048] PCI host bridge to bus 0000:00 [ 0.408014] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.410014] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.412015] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.415020] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.418023] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.420030] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.422185] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.424904] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.427190] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.432743] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.435964] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.438023] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.439008] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.442018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.443651] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.447884] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.450116] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.453943] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.457787] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.466025] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.470013] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.473000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.478014] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.481997] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.492014] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.500473] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.506018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.511017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.530893] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.540449] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.542266] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.545326] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.547482] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.549179] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.554144] iommu: Default domain type: Passthrough [ 0.555463] SCSI subsystem initialized [ 0.556099] ACPI: bus type USB registered [ 0.557079] usbcore: registered new interface driver usbfs [ 0.558043] usbcore: registered new interface driver hub [ 0.559068] usbcore: registered new device driver usb [ 0.560083] pps_core: LinuxPPS API ver. 1 registered [ 0.561007] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.563065] PTP clock support registered [ 0.565054] EDAC MC: Ver: 3.0.0 [ 0.566076] PCI: Using ACPI for IRQ routing [ 0.567419] NetLabel: Initializing [ 0.569007] NetLabel: domain hash size = 128 [ 0.569917] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.571065] NetLabel: unlabeled traffic allowed by default [ 0.572152] vgaarb: loaded [ 0.573281] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.574010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.578322] clocksource: Switched to clocksource kvm-clock [ 0.669615] VFS: Disk quotas dquot_6.6.0 [ 0.670826] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.672462] *** VALIDATE ramfs *** [ 0.673254] *** VALIDATE hugetlbfs *** [ 0.674052] pnp: PnP ACPI init [ 0.676468] pnp: PnP ACPI: found 6 devices [ 0.689952] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.692337] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.694160] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.696022] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.697861] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.699582] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.701495] NET: Registered protocol family 2 [ 0.703558] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.707321] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.711742] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.717673] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.721919] TCP: Hash tables configured (established 65536 bind 65536) [ 0.725595] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.728633] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.731338] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.733983] NET: Registered protocol family 1 [ 0.736287] RPC: Registered named UNIX socket transport module. [ 0.738347] RPC: Registered udp transport module. [ 0.739810] RPC: Registered tcp transport module. [ 0.741362] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.743630] NET: Registered protocol family 44 [ 0.745091] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.746650] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.748218] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.749996] PCI: CLS 0 bytes, default 64 [ 0.751226] Unpacking initramfs... [ 2.031126] debug: unmapping init [mem 0xffff96f97cc64000-0xffff96f97ffcffff] [ 2.035274] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.037522] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.040576] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.525264] Initialise system trusted keyrings [ 2.527023] Key type blacklist registered [ 2.528399] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.537441] zbud: loaded [ 2.540420] *** VALIDATE nfs *** [ 2.541273] *** VALIDATE nfs4 *** [ 2.542517] pstore: using deflate compression [ 2.545504] Platform Keyring initialized [ 2.639082] NET: Registered protocol family 38 [ 2.640654] Key type asymmetric registered [ 2.641863] Asymmetric key parser 'x509' registered [ 2.643290] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.645546] io scheduler mq-deadline registered [ 2.646767] io scheduler kyber registered [ 2.647995] io scheduler bfq registered [ 2.649491] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.651797] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.654316] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.656923] ACPI: Power Button [PWRF] [ 2.661909] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.667803] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.682770] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.712116] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.739567] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.744405] Non-volatile memory driver v1.3 [ 2.745921] Linux agpgart interface v0.103 [ 2.771062] virtio_blk virtio1: [vda] 139280 512-byte logical blocks (71.3 MB/68.0 MiB) [ 2.774131] vda: detected capacity change from 0 to 71311360 [ 2.787994] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.790840] vdb: detected capacity change from 0 to 1073741824 [ 2.795793] libphy: Fixed MDIO Bus: probed [ 2.800672] usbcore: registered new interface driver usbserial_generic [ 2.803257] usbserial: USB Serial support registered for generic [ 2.805220] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.809308] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.811039] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.812724] mousedev: PS/2 mouse device common for all mice [ 2.814798] rtc_cmos 00:05: RTC can wake from S4 [ 2.816807] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.817637] rtc_cmos 00:05: registered as rtc0 [ 2.820255] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.821851] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.823661] intel_pstate: CPU model not supported [ 2.829764] hid: raw HID events driver (C) Jiri Kosina [ 2.832256] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.832721] usbcore: registered new interface driver usbhid [ 2.837747] usbhid: USB HID core driver [ 2.839261] drop_monitor: Initializing network drop monitor service [ 2.841554] Initializing XFRM netlink socket [ 2.843339] NET: Registered protocol family 10 [ 2.846032] Segment Routing with IPv6 [ 2.847590] NET: Registered protocol family 17 [ 2.849509] mpls_gso: MPLS GSO support [ 2.854946] RAS: Correctable Errors collector initialized. [ 2.858078] AVX version of gcm_enc/dec engaged. [ 2.859851] AES CTR mode by8 optimization enabled [ 2.926088] sched_clock: Marking stable (2926053968, 0)->(3680840827, -754786859) [ 2.929105] registered taskstats version 1 [ 2.930661] Loading compiled-in X.509 certificates [ 2.932982] zswap: loaded using pool lzo/zbud [ 2.954968] Key type big_key registered [ 2.966551] Key type encrypted registered [ 2.968407] ima: No TPM chip found, activating TPM-bypass! [ 2.970580] ima: Allocated hash algorithm: sha1 [ 2.972485] ima: No architecture policies found [ 2.974381] evm: Initialising EVM extended attributes: [ 2.976395] evm: security.selinux [ 2.977646] evm: security.ima [ 2.978750] evm: security.capability [ 2.980180] evm: HMAC attrs: 0x1 [ 2.982354] rtc_cmos 00:05: setting system clock to 2026-06-17 01:23:59 UTC (1781659439) [ 2.988569] debug: unmapping init [mem 0xffffffffbd603000-0xffffffffbd7fffff] [ 2.991578] debug: unmapping init [mem 0xffffffffbc382000-0xffffffffbc658fff] [ 3.004060] Write protecting the kernel read-only data: 28672k [ 3.007443] debug: unmapping init [mem 0xffffffffbaa03000-0xffffffffbabfffff] [ 3.009704] debug: unmapping init [mem 0xffffffffbb314000-0xffffffffbb3fffff] [ 3.036428] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.044191] systemd[1]: Detected virtualization kvm. [ 3.045870] systemd[1]: Detected architecture x86-64. [ 3.047283] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.070824] systemd[1]: No hostname configured. [ 3.072645] systemd[1]: Set hostname to . [ 3.074761] random: systemd: uninitialized urandom read (16 bytes read) [ 3.076564] systemd[1]: Initializing machine ID from random generator. [ 3.178618] random: systemd: uninitialized urandom read (16 bytes read) [ 3.182693] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.187293] random: systemd: uninitialized urandom read (16 bytes read) [ 3.189708] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.194205] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Reached target Timers. Starting Journal Service... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Setup Virtual Console... [ OK ] Reached target Swap. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook...[ 3.731893] random: fast init done [ 3.774295] device-mapper: uevent: version 1.0.3 [ 3.775976] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.482419] virtio_net virtio0 ens2: renamed from eth0 [ 4.567082] scsi host0: ata_piix [ 4.582746] scsi host1: ata_piix [ 4.584364] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 4.586452] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 8.897458] dracut-initqueue[586]: RTNETLINK answers: File exists [ 9.500583] random: crng init done [ 9.501950] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 9.811452] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 10.851333] printk: systemd: 25 output lines suppressed due to ratelimiting [ 11.053289] SELinux: Disabled at runtime. [ 11.110316] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.116968] systemd[1]: Detected virtualization kvm. [ 11.118958] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.551941] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.554907] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.562387] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.569856] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.572919] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.578963] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.583193] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Remount Root and Kernel File Systems... Mounting POSIX Message Queue File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice User and Session Slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Huge Pages File System... [ OK ] Listening on udev Kernel Socket. Mounting Kernel Debug File System... Starting Apply Kernel Variables... [ OK ] Reached target Slices. [ OK ] Listening on initctl Compatibility Named Pipe. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ 11.712765] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 11.951090] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.274337] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.301486] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.392692] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.401704] EDAC sbridge: Ver: 1.1.2 [ 13.372133] Key type dns_resolver registered [ 13.656037] NFS: Registering the id_resolver key type [ 13.657516] Key type id_resolver registered [ 13.658329] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg452-client login: [ 36.951581] libcfs: loading out-of-tree module taints kernel. [ 37.017244] Key type ._llcrypt registered [ 37.018309] Key type .llcrypt registered [ 37.219855] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 37.224757] alg: No test for adler32 (adler32-zlib) [ 38.199870] Lustre: Lustre: Build Version: 2.17.54_40_g6e23802 [ 38.508116] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 40.128196] Key type lgssc registered [ 40.673411] Lustre: Echo OBD driver; http://www.lustre.org/ [ 84.497485] Lustre: Mounted lustre-client [ 86.763616] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 95.225722] Lustre: DEBUG MARKER: oleg452-client.virtnet: executing check_logdir /tmp/testlogs/ [ 96.753286] Lustre: DEBUG MARKER: oleg452-client.virtnet: executing yml_node [ 98.097619] Lustre: DEBUG MARKER: Client: 2.17.54.40 [ 98.930462] Lustre: DEBUG MARKER: MDS: 2.17.54.40 [ 99.774916] Lustre: DEBUG MARKER: OSS: 2.17.54.40 [ 100.324269] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Tue Jun 16 21:25:36 EDT 2026 [ 104.873707] Lustre: Unmounted lustre-client [ 112.159445] Key type lgssc unregistered [ 112.286310] LNet: 8381:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 112.288528] LNetError: 8381:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 112.299424] LNet: Removed LNI 192.168.204.52@tcp [ 112.552092] Key type .llcrypt unregistered [ 112.553403] Key type ._llcrypt unregistered [ 120.422798] Key type ._llcrypt registered [ 120.423779] Key type .llcrypt registered [ 120.475877] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 120.481157] alg: No test for adler32 (adler32-zlib) [ 121.283583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 121.310892] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 121.319090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show [ 122.559465] Lustre: DEBUG MARKER: excepting tests: 253 254 410 [ 122.583344] LNet: 9255:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 122.585785] LNetError: 9255:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 122.593486] LNet: Removed LNI 192.168.204.52@tcp [ 122.821338] Key type .llcrypt unregistered [ 122.822341] Key type ._llcrypt unregistered [ 124.830214] Key type ._llcrypt registered [ 124.831553] Key type .llcrypt registered [ 125.296553] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 21:26:01 (1781659561) [ 125.417140] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 125.423161] alg: No test for adler32 (adler32-zlib) [ 126.231932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 126.261112] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 126.269242] LNet: 9672:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 126.277762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 126.302477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 126.311515] LNet: 9679:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 128.646654] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 21:26:04 (1781659564) [ 128.677116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 128.689232] LNet: 10249:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 128.702162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 128.742272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp [ 131.040506] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 21:26:07 (1781659567) [ 131.080804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 131.096268] LNet: 10837:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 131.111467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 131.149221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 2.2.2.2@o2ib [ 133.526922] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 21:26:09 (1781659569) [ 133.560080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 133.571217] LNet: 11418:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 133.582220] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 133.615563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib [ 135.887938] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 21:26:11 (1781659571) [ 135.930293] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 135.948321] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 135.979631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp [ 136.127747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp [ 138.299035] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 21:26:14 (1781659574) [ 138.336022] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 138.346143] LNet: 12592:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 138.349951] LNet: 12592:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 138.365712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 138.399299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 138.554236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 138.691052] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib [ 140.761316] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 21:26:16 (1781659576) [ 140.793518] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 140.814881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 140.853529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi,[6-12/4]@bxi3f [ 143.007508] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 21:26:19 (1781659579) [ 143.035292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 143.050936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 143.078843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp [ 143.090926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 143.229221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib [ 143.242494] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 143.373042] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp [ 143.388626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 143.521695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib [ 143.535437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 143.668594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib [ 143.681112] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 143.831354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni [ 143.845415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 143.992048] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni [ 144.008579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 144.142075] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi [ 144.157531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 144.307039] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi [ 144.324187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 144.466186] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f [ 144.481863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 144.625677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f --nid [8-12]@bxi3f [ 144.642807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 144.791774] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi,[3-7]@bxi3f [ 144.808445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 144.960130] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 144.975506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 147.135490] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 21:26:23 (1781659583) [ 147.167527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 147.176518] LNet: 14450:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 147.179245] LNet: 14450:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 147.189593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 147.219952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp [ 147.233677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp [ 149.456484] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 21:26:25 (1781659585) [ 149.483704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 149.501537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 149.531733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 149.545663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 151.840544] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 21:26:27 (1781659587) [ 151.878221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 151.904621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 151.945609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp [ 151.963918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp [ 154.275296] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 21:26:30 (1781659590) [ 154.302837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 154.320571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 154.354322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 154.370382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 156.569123] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 21:26:32 (1781659592) [ 156.609779] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 156.627477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 156.666164] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp [ 156.682867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp [ 158.790639] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 21:26:34 (1781659594) [ 158.822712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 158.839605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 158.871523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 158.886482] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 161.022437] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 21:26:37 (1781659597) [ 161.063136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 161.082139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 161.122347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 161.264276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 163.411424] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 21:26:39 (1781659599) [ 163.448977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 163.460281] LNet: 18558:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 163.463181] LNet: 18558:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 6 previous similar messages [ 163.475296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 163.540187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 163.618250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 165.932556] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 21:26:41 (1781659601) [ 165.966726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 165.986443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 166.035882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 166.109242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 168.230289] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 21:26:44 (1781659604) [ 168.259555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 168.277628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 168.452661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 168.551437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 170.728755] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 21:26:46 (1781659606) [ 170.757830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 170.774439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 170.806762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 170.824106] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 172.936918] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 21:26:48 (1781659608) [ 172.970592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 172.991034] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 173.024070] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 173.041850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 175.300956] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 21:26:51 (1781659611) [ 175.330577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 175.346807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 175.377917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 19@gni [ 177.510959] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 21:26:53 (1781659613) [ 177.548247] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 177.564618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 177.594071] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib [ 179.734655] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 21:26:55 (1781659615) [ 179.769513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 179.789650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 179.829843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni [ 179.965207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni [ 182.068187] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 21:26:58 (1781659618) [ 182.097132] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 182.112762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 182.147263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni [ 182.161593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 22@gni --nid 26@gni [ 184.435693] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 21:27:00 (1781659620) [ 184.470170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 184.489061] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 184.524763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni [ 184.537786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni [ 186.751937] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 21:27:02 (1781659622) [ 186.786597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 186.804233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 186.841424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni [ 186.858165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp [ 189.143410] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 21:27:05 (1781659625) [ 189.192651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 189.210580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 189.241546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 189.257810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 191.552275] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 21:27:07 (1781659627) [ 191.580762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 191.596411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 191.622331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim [ 191.647654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp [ 191.655616] LNetError: 25756:0:(peer.c:2175:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 191.671034] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --force [ 191.685738] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show --nid 1.1.1.1@tcp [ 193.748195] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 21:27:09 (1781659629) [ 193.783587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 193.802096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 193.829935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.rADBIf9pVq [ 193.857247] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.rADBIf9pVq [ 193.879800] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.rADBIf9pVq [ 195.967859] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 21:27:11 (1781659631) [ 196.000851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 196.010972] LNet: 26922:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 196.013393] LNet: 26922:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 13 previous similar messages [ 196.025200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 196.047141] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp [ 196.061953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp [ 196.078379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer list [ 198.244430] Lustre: DEBUG MARKER: == sanity-lnet test 50: Enable/disable routing via yaml import ========================================================== 21:27:14 (1781659634) [ 198.273837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 198.294105] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 198.331848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 199.328505] LNet: Message forwarding will be enabled in 2s [ 202.400116] LNet: Message forwarding enabled [ 202.482383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 202.492206] LNet: Message forwarding will stop in 2s to 6s [ 206.666676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 207.289493] LNet: Message forwarding disabled [ 207.793953] LNet: Message forwarding will be enabled in 2s [ 210.020895] Lustre: DEBUG MARKER: == sanity-lnet test 51: Set numa range via yaml import === 21:27:26 (1781659646) [ 210.052838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 210.637870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 210.672966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-51.yaml [ 212.783923] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 21:27:28 (1781659648) [ 212.822765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 212.843739] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 212.871724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid foobar [ 212.886442] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid foobar [ 212.901862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@o2ib [ 212.915121] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --nid 1.1.1.1@tcp [ 212.931674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --nid 1.1.1.1@tcp [ 212.947859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add [ 212.964427] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 212.982225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp [ 212.996695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@foo [ 213.010198] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@tcp [ 213.023557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@o2ib [ 213.039598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@gni [ 213.055735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni [ 213.069449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni [ 213.085817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp [ 213.101393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp [ 213.117398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp [ 213.132760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 213.148952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp --gateway 1@gni [ 213.159484] LNetError: 28821:0:(router.c:624:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni [ 215.146945] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 21:27:31 (1781659651) [ 215.179343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 215.198407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 215.238133] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 217.287890] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 21:27:33 (1781659653) [ 217.888042] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_100 Need at least 2 remote nodes found oleg452-server [ 218.510611] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 21:27:34 (1781659654) [ 218.553787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 218.576472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 218.607970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 218.618697] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 219.212090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp1 --gateway 192.168.204.[53-57]@tcp [ 219.221132] LNet: 30535:0:(router.c:728:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 219.230572] LNetError: 30324:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.204.55@tcp is being used as a gateway but routing feature is not turned on [ 219.237843] LNetError: 30324:0:(lib-lnet.h:1267:lnet_set_route_aliveness()) route to tcp1 through 192.168.204.55@tcp has gone from up to down [ 220.257857] LNetError: 30324:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.204.55@tcp is being used as a gateway but routing feature is not turned on [ 220.262707] LNetError: 30324:0:(router.c:393:lnet_router_discovery_ping_reply()) Skipped 3 previous similar messages [ 221.281639] LNetError: 30324:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.204.56@tcp is being used as a gateway but routing feature is not turned on [ 221.284964] LNetError: 30324:0:(router.c:393:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message [ 221.360273] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 21:27:37 (1781659657) [ 221.975262] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_102 Need at least 2 remote nodes found oleg452-server [ 222.305428] LNetError: 30330:0:(lib-lnet.h:1267:lnet_set_route_aliveness()) route to tcp1 through 192.168.204.54@tcp has gone from up to down [ 222.310570] LNetError: 30330:0:(lib-lnet.h:1267:lnet_set_route_aliveness()) Skipped 1 previous similar message [ 222.563621] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 21:27:38 (1781659658) [ 222.596214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 222.607346] LNetError: 31464:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 222.619168] LNet: Removed LNI 192.168.204.52@tcp [ 222.629930] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 222.654799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 222.665373] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 222.935535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp103 --gateway 192.168.204.[53-58/2]@tcp [ 222.944574] LNet: 31586:0:(router.c:728:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 222.947261] LNet: 31586:0:(router.c:728:lnet_add_route()) Skipped 4 previous similar messages [ 222.953730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route del --net tcp103 --gateway 192.168.204.[53-58/2]@tcp [ 225.109094] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 21:27:41 (1781659661) [ 225.147831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.208683] LNetError: 32175:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 226.217068] LNet: Removed LNI 192.168.204.52@tcp [ 226.227113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.260436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking -1 [ 226.282823] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.295994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.318340] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 226.342162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.357243] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.379543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 0 [ 226.411404] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.427530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.453697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 226.479478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.492628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.512838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 1 [ 226.548617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.563795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.586778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 226.616577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.630345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.652387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 2 [ 226.684629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.698820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.725494] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 226.760725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.774520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.794937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 226.823203] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.837601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.862551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 226.890534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.904577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.924216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 4 [ 226.944732] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.959205] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.984673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 229.012689] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 21:27:45 (1781659665) [ 229.571224] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_105 Need at least 2 remote nodes found oleg452-server [ 230.089171] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 21:27:46 (1781659666) [ 230.654415] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_106 Need at least 2 remote nodes found oleg452-server [ 231.163820] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 21:27:47 (1781659667) [ 231.437331] Key type .llcrypt unregistered [ 231.438421] Key type ._llcrypt unregistered [ 233.333509] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 233.412845] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 233.414889] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 233.417135] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 235.579963] Key type ._llcrypt registered [ 235.581127] Key type .llcrypt registered [ 235.639933] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 235.645359] alg: No test for adler32 (adler32-zlib) [ 236.442877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 236.461980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 236.471712] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 236.671882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 236.678549] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 237.353037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 237.361312] LNet: Removed LNI 10.1.2.3@tcp [ 237.402571] LNet: 34297:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 237.405272] LNetError: 34297:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 237.413304] LNet: Removed LNI 192.168.204.52@tcp [ 237.650415] Key type .llcrypt unregistered [ 237.652195] Key type ._llcrypt unregistered [ 241.141949] Key type ._llcrypt registered [ 241.143024] Key type .llcrypt registered [ 241.585678] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 21:27:57 (1781659677) [ 241.755358] Key type .llcrypt unregistered [ 241.756241] Key type ._llcrypt unregistered [ 243.641614] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 243.690311] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 243.693213] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 243.695465] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 245.890726] Key type ._llcrypt registered [ 245.891864] Key type .llcrypt registered [ 245.949946] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 245.956320] alg: No test for adler32 (adler32-zlib) [ 246.757683] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 246.778459] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 246.786840] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 247.012437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 247.020201] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 247.776120] LNet: 35789:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 247.778193] LNetError: 35789:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 247.779961] LNet: Removed LNI 192.168.204.52@tcp [ 247.785257] LNet: Removed LNI 10.1.2.3@tcp [ 247.973411] Key type .llcrypt unregistered [ 247.975259] Key type ._llcrypt unregistered [ 251.409758] Key type ._llcrypt registered [ 251.410704] Key type .llcrypt registered [ 251.831046] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 21:28:07 (1781659687) [ 251.992316] Key type .llcrypt unregistered [ 251.993401] Key type ._llcrypt unregistered [ 253.910454] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 253.997450] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 253.998987] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 254.000398] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 256.201072] Key type ._llcrypt registered [ 256.201906] Key type .llcrypt registered [ 256.258429] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 256.264035] alg: No test for adler32 (adler32-zlib) [ 257.060084] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 257.080151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 257.089530] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 257.710101] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 257.718367] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 258.382023] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 258.390971] LNet: Removed LNI 10.1.2.3@tcp [ 258.397131] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg:0 [ 258.417326] LNet: Removed LNI 10.1.2.31@tcp [ 258.423771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 258.433311] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 258.680484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 258.689538] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 259.004872] LNet: 37718:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 259.008088] LNetError: 37718:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 259.010791] LNet: Removed LNI 10.1.2.3@tcp [ 259.015354] LNet: Removed LNI 10.1.2.31@tcp [ 259.235525] Key type .llcrypt unregistered [ 259.237064] Key type ._llcrypt unregistered [ 262.702536] Key type ._llcrypt registered [ 262.704483] Key type .llcrypt registered [ 263.149883] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 21:28:19 (1781659699) [ 263.266041] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 263.271516] alg: No test for adler32 (adler32-zlib) [ 264.071860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 264.091803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 264.100496] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 264.379905] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 264.388245] LNet: 38668:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 264.391191] LNetError: 38668:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 264.400372] LNet: Removed LNI 192.168.204.52@tcp [ 264.408955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 264.430227] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --nid 192.168.204.52@tcp [ 264.440108] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 264.690805] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --nid 192.168.204.52@tcp [ 264.705514] LNet: Removed LNI 192.168.204.52@tcp [ 264.951531] Key type .llcrypt unregistered [ 264.953207] Key type ._llcrypt unregistered [ 268.466976] Key type ._llcrypt registered [ 268.468277] Key type .llcrypt registered [ 268.907708] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 21:28:24 (1781659704) [ 269.501758] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_111 Need at least 2 remote nodes found oleg452-server [ 270.050678] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 21:28:26 (1781659706) [ 270.229384] Key type .llcrypt unregistered [ 270.230873] Key type ._llcrypt unregistered [ 272.121729] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 272.153765] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 272.156187] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 272.158808] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 274.358845] Key type ._llcrypt registered [ 274.360401] Key type .llcrypt registered [ 274.423286] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 274.428850] alg: No test for adler32 (adler32-zlib) [ 275.230826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 275.252354] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 275.506571] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180] [ 276.198821] LNet: 40635:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 276.202367] LNetError: 40635:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 276.206395] LNet: Removed LNI 192.168.204.52@tcp1 [ 276.219282] LNet: Removed LNI 10.1.2.3@tcp2 [ 276.433290] Key type .llcrypt unregistered [ 276.434462] Key type ._llcrypt unregistered [ 279.869746] Key type ._llcrypt registered [ 279.870730] Key type .llcrypt registered [ 280.284352] Lustre: DEBUG MARKER: == sanity-lnet test 150: Check import of ip2nets yaml sequence ========================================================== 21:28:36 (1781659716) [ 280.393728] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 280.399210] alg: No test for adler32 (adler32-zlib) [ 281.198908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 281.218597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 281.228575] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 281.484581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 281.492842] LNet: 41602:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 281.495213] LNetError: 41602:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 281.504328] LNet: Removed LNI 192.168.204.52@tcp [ 281.511617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 281.532419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet.yaml [ 281.541459] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 283.540324] Lustre: DEBUG MARKER: == sanity-lnet test 151: Check correct application of multiple ip2nets rules ========================================================== 21:28:39 (1781659719) [ 283.574464] LNet: 42187:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 283.577819] LNetError: 42187:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 283.585430] LNet: Removed LNI 192.168.204.52@tcp [ 283.777377] Key type .llcrypt unregistered [ 283.778688] Key type ._llcrypt unregistered [ 285.606370] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 285.645303] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 285.647658] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 285.650100] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 287.844331] Key type ._llcrypt registered [ 287.845574] Key type .llcrypt registered [ 287.901882] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 287.906799] alg: No test for adler32 (adler32-zlib) [ 288.705914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 288.727401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 288.739213] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 288.994963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 289.002747] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 289.694900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 289.703481] LNet: 42935:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 289.705966] LNetError: 42935:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 289.708471] LNet: Removed LNI 192.168.204.52@tcp [ 289.713245] LNet: Removed LNI 10.1.2.3@tcp [ 289.720160] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 289.744072] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 289.754480] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 289.757610] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 289.905585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 289.913471] LNet: Removed LNI 192.168.204.52@tcp [ 289.921249] LNet: Removed LNI 10.1.2.3@tcp [ 289.928426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 289.950126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 289.960796] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 289.962931] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 290.103207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 290.112358] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 290.462348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 290.470271] LNet: 43125:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 290.471991] LNet: 43125:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 290.474364] LNetError: 43125:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 290.476346] LNetError: 43125:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 290.478818] LNet: Removed LNI 192.168.204.52@tcp [ 290.479773] LNet: Removed LNI 10.1.2.3@tcp [ 290.487451] LNet: Removed LNI 192.168.204.52@tcp2 [ 290.494657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 290.518490] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 290.528202] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 290.530329] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 290.533283] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 292.617737] Lustre: DEBUG MARKER: == sanity-lnet test 152: Check import of ip2nets with wildcard IP patterns ========================================================== 21:28:48 (1781659728) [ 292.647295] LNet: 43714:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 292.650467] LNetError: 43714:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 292.652910] LNetError: 43714:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 292.655577] LNet: Removed LNI 192.168.204.52@tcp [ 292.657157] LNet: Removed LNI 10.1.2.3@tcp [ 292.663480] LNet: Removed LNI 192.168.204.52@tcp2 [ 292.873666] Key type .llcrypt unregistered [ 292.875323] Key type ._llcrypt unregistered [ 294.630418] Key type ._llcrypt registered [ 294.631392] Key type .llcrypt registered [ 294.692674] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 294.698111] alg: No test for adler32 (adler32-zlib) [ 295.496169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 295.515461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 295.523565] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 295.787547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 295.796773] LNet: 44114:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 295.799486] LNetError: 44114:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 295.807262] LNet: Removed LNI 192.168.204.52@tcp [ 295.814294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 295.838280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 295.848349] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 296.009015] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 296.021361] LNet: Removed LNI 192.168.204.52@tcp [ 296.029710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 296.053412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 296.062980] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 296.206157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 296.219438] LNet: Removed LNI 192.168.204.52@tcp [ 296.226887] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 296.247654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 296.255615] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 296.398800] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 296.442344] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 296.443665] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 296.445518] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 298.580062] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 298.586612] LNet: 44253:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 298.588448] LNet: 44253:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 298.590317] LNetError: 44253:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 298.592900] LNetError: 44253:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 298.601301] LNet: Removed LNI 192.168.204.52@tcp [ 298.609230] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 298.631240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 298.639828] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 300.748472] Lustre: DEBUG MARKER: == sanity-lnet test 153: Check import of ip2nets with IP address ranges ========================================================== 21:28:56 (1781659736) [ 300.779543] LNet: 44843:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 300.783303] LNetError: 44843:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 300.793154] LNet: Removed LNI 192.168.204.52@tcp [ 301.005685] Key type .llcrypt unregistered [ 301.006887] Key type ._llcrypt unregistered [ 302.861557] Key type ._llcrypt registered [ 302.862707] Key type .llcrypt registered [ 302.924399] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 302.930156] alg: No test for adler32 (adler32-zlib) [ 303.727948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 303.746139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 303.754085] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 304.019828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 304.027382] LNet: 45247:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 304.029233] LNetError: 45247:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 304.043285] LNet: Removed LNI 192.168.204.52@tcp [ 304.050811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 304.071167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-153.yaml [ 304.079713] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 306.065285] Lustre: DEBUG MARKER: == sanity-lnet test 154: Check import of ip2nets with stepped IP address ranges ========================================================== 21:29:02 (1781659742) [ 306.093764] LNet: 45830:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 306.096133] LNetError: 45830:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 306.102455] LNet: Removed LNI 192.168.204.52@tcp [ 306.305400] Key type .llcrypt unregistered [ 306.306403] Key type ._llcrypt unregistered [ 308.098638] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 308.130934] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 308.132412] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 308.135106] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 310.319844] Key type ._llcrypt registered [ 310.320945] Key type .llcrypt registered [ 310.373674] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 310.378589] alg: No test for adler32 (adler32-zlib) [ 311.176860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 311.198511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 311.208062] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 311.489955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 311.497038] LNet: 46287:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 311.499760] LNetError: 46287:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 311.509264] LNet: Removed LNI 192.168.204.52@tcp [ 311.517371] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 311.535943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-154.yaml [ 311.543735] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 313.623465] Lustre: DEBUG MARKER: == sanity-lnet test 155: Check ip2nets import failure with no matching rules ========================================================== 21:29:09 (1781659749) [ 313.651440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 313.662282] LNet: 46879:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 313.665495] LNetError: 46879:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 313.673687] LNet: Removed LNI 192.168.204.52@tcp [ 313.683591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 313.719399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-155.yaml [ 315.472803] Lustre: DEBUG MARKER: == sanity-lnet test 156: Check import of ip2nets with comma-separated IP ranges ========================================================== 21:29:11 (1781659751) [ 315.508662] LNet: 47463:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 315.719475] Key type .llcrypt unregistered [ 315.720826] Key type ._llcrypt unregistered [ 317.569624] Key type ._llcrypt registered [ 317.570682] Key type .llcrypt registered [ 317.624493] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 317.629816] alg: No test for adler32 (adler32-zlib) [ 318.422410] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 318.439161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 318.447634] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 318.702679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 318.709182] LNet: 47862:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 318.711307] LNetError: 47862:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 318.720359] LNet: Removed LNI 192.168.204.52@tcp [ 318.727902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 318.748878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-156.yaml [ 318.758510] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 320.702517] Lustre: DEBUG MARKER: == sanity-lnet test 157: Check import failure with malformed ip2nets YAML ========================================================== 21:29:16 (1781659756) [ 320.734035] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 320.742180] LNet: 48451:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 320.744480] LNetError: 48451:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 320.754459] LNet: Removed LNI 192.168.204.52@tcp [ 320.762938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 320.788400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-157.yaml [ 322.636571] Lustre: DEBUG MARKER: == sanity-lnet test 158: Check import of ip2nets with single IPv6 address ========================================================== 21:29:18 (1781659758) [ 323.092642] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_158 Interface has no IPv6 [ 323.625753] Lustre: DEBUG MARKER: == sanity-lnet test 159: Check import of ip2nets with IPv6 /128 CIDR notation ========================================================== 21:29:19 (1781659759) [ 324.108604] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_159 Interface has no IPv6 [ 324.623967] Lustre: DEBUG MARKER: == sanity-lnet test 160: Check correct application of multiple IPv6 ip2nets rules ========================================================== 21:29:20 (1781659760) [ 325.117931] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_160 Interface has no IPv6 [ 325.709121] Lustre: DEBUG MARKER: == sanity-lnet test 161: Check import of ip2nets with various IPv6 prefix lengths ========================================================== 21:29:21 (1781659761) [ 326.196451] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_161 Interface has no IPv6 [ 326.755869] Lustre: DEBUG MARKER: == sanity-lnet test 162: Check ip2nets import failure with non-matching IPv6 ========================================================== 21:29:22 (1781659762) [ 327.227859] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_162 Interface has no IPv6 [ 327.743846] Lustre: DEBUG MARKER: == sanity-lnet test 170: Check CPTs and tunables parsing when importing ========================================================== 21:29:23 (1781659763) [ 327.775534] LNet: 50496:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 328.004851] Key type .llcrypt unregistered [ 328.006059] Key type ._llcrypt unregistered [ 329.906582] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 329.956138] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 329.958429] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 329.960820] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 332.153268] Key type ._llcrypt registered [ 332.154549] Key type .llcrypt registered [ 332.215481] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 332.221653] alg: No test for adler32 (adler32-zlib) [ 333.022627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 333.047157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 333.054806] LNet: Added LNI 192.168.204.52@tcp [8/512/0/180] [ 333.056700] LNet: Added LNI 10.1.2.3@tcp [8/512/0/180] [ 333.058202] LNet: 50822:0:(api-ni.c:2653:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 333.091269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 333.098907] LNet: 50841:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 333.101126] LNetError: 50841:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 333.104637] LNet: Removed LNI 192.168.204.52@tcp [ 333.106100] LNetError: 50841:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 333.119321] LNet: Removed LNI 10.1.2.3@tcp [ 333.127709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 333.154202] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 333.164627] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 333.188824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 333.197316] LNet: 50865:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 333.200795] LNetError: 50865:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 333.214317] LNet: Removed LNI 192.168.204.52@tcp [ 333.221760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 333.242860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 333.252884] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 333.256211] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 333.257892] LNet: 50873:0:(api-ni.c:2653:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 333.286691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 333.294929] LNet: 50893:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 333.297307] LNetError: 50893:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 333.300040] LNet: Removed LNI 192.168.204.52@tcp [ 333.301613] LNetError: 50893:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 333.308303] LNet: Removed LNI 10.1.2.3@tcp [ 333.315501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 333.336943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 333.345242] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 333.347297] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 333.348526] LNet: 50901:0:(api-ni.c:2653:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 333.402604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 333.411155] LNet: 50927:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 333.414239] LNetError: 50927:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 333.416917] LNet: Removed LNI 192.168.204.52@tcp [ 333.418354] LNetError: 50927:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 333.429327] LNet: Removed LNI 10.1.2.3@tcp [ 333.436340] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 333.459692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 333.469264] LNet: Added LNI 192.168.204.52@tcp [8/256/0/182] [ 333.471513] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 333.501475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 333.508617] LNet: 50955:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 333.512211] LNetError: 50955:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 333.516058] LNet: Removed LNI 192.168.204.52@tcp [ 333.517352] LNetError: 50955:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 333.525287] LNet: Removed LNI 10.1.2.3@tcp [ 333.533942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 333.558360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 333.566726] LNet: Added LNI 192.168.204.52@tcp [8/256/0/182] [ 333.569395] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 333.571089] LNet: 50963:0:(api-ni.c:2653:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 333.599531] LNet: Removed LNI 192.168.204.52@tcp [ 333.610428] LNet: Removed LNI 10.1.2.3@tcp [ 333.829914] Key type .llcrypt unregistered [ 333.831034] Key type ._llcrypt unregistered [ 337.326160] Key type ._llcrypt registered [ 337.327632] Key type .llcrypt registered [ 337.735652] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 21:29:33 (1781659773) [ 337.846944] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 337.852645] alg: No test for adler32 (adler32-zlib) [ 338.647876] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 338.662763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 338.670494] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 338.940024] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --ip2net tcp 192.168.204.52 [ 338.947589] LNet: 51938:0:(config.c:1627:lnet_parse_ip2nets()) Kernel parsing of ip2nets is deprecated. Consider converting to LNet YAML configuration. [ 338.951783] LNet: Removed LNI 192.168.204.52@tcp [ 340.801588] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 21:29:36 (1781659776) [ 340.839032] LNet: 52502:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 340.841848] LNetError: 52502:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 340.850672] LNet: Removed LNI 192.168.204.52@tcp [ 341.054419] Key type .llcrypt unregistered [ 341.055731] Key type ._llcrypt unregistered [ 342.856906] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 342.952068] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 345.208536] Key type ._llcrypt registered [ 345.210151] Key type .llcrypt registered [ 345.264847] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 345.271713] alg: No test for adler32 (adler32-zlib) [ 346.201438] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 348.073203] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 21:29:44 (1781659784) [ 348.100757] LNet: 53413:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 348.103684] LNetError: 53413:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 348.112678] LNet: Removed LNI 10.1.2.3@tcp [ 348.323534] Key type .llcrypt unregistered [ 348.324867] Key type ._llcrypt unregistered [ 350.118314] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 350.222227] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 350.223860] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 350.225789] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 352.489706] Key type ._llcrypt registered [ 352.490996] Key type .llcrypt registered [ 352.544446] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 352.549678] alg: No test for adler32 (adler32-zlib) [ 353.371253] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 355.335305] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 21:29:51 (1781659791) [ 355.794649] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_202 kernel ip2nets deprecated in 2.17.53 [ 356.288906] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 21:29:52 (1781659792) [ 356.322727] LNet: 54597:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 356.324990] LNetError: 54597:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 356.332534] LNet: Removed LNI 10.1.2.3@tcp [ 356.543491] Key type .llcrypt unregistered [ 356.545318] Key type ._llcrypt unregistered [ 358.318463] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 358.412524] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 358.413892] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 358.415435] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 360.672079] Key type ._llcrypt registered [ 360.673371] Key type .llcrypt registered [ 360.729652] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 360.735413] alg: No test for adler32 (adler32-zlib) [ 361.528978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 361.554757] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 364.083836] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 21:30:00 (1781659800) [ 364.210524] LNet: 55829:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 364.212835] LNetError: 55829:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 364.219501] LNet: Removed LNI 10.1.2.3@tcp [ 364.414484] Key type .llcrypt unregistered [ 364.415908] Key type ._llcrypt unregistered [ 366.256571] Key type ._llcrypt registered [ 366.257936] Key type .llcrypt registered [ 366.321389] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 366.328032] alg: No test for adler32 (adler32-zlib) [ 367.126443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 367.135232] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 369.873059] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 370.063689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_interrupt -r 1 [ 370.074625] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_interrupt -r 1 [ 370.088753] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 370.095645] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 370.166333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 370.177523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 370.187501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 370.197813] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 370.208782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 370.210953] LNet: 1 local NIs in recovery (showing 1): 192.168.204.52@tcp [ 370.219557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 370.231058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 370.242113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 370.250478] LNet: 56265:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 370.257463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 370.267838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 370.279341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 370.290665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 370.302652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 370.314161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 370.325838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 370.484065] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_dropped -r 1 [ 370.497598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_dropped -r 1 [ 370.510958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 370.583757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 370.596430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 370.607976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 370.619934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 370.631074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 370.642839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 370.655121] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 370.667297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 370.679963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 370.692272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 370.704362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 370.717377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 370.729268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 370.741093] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 370.753701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 370.911076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_aborted -r 1 [ 370.923473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_aborted -r 1 [ 370.935786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 370.942801] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 370.944508] LNet: Skipped 1 previous similar message [ 371.013330] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 371.025153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 371.036698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 371.047501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 371.058423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 371.069305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 371.080863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 371.093776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 371.101941] LNet: 56484:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 371.105533] LNet: 56484:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 371.114093] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 371.128465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 371.143085] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 371.155145] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 371.168455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 371.183037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 371.197142] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 371.355223] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_no_route -r 1 [ 371.368238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_no_route -r 1 [ 371.382813] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 371.457389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 371.469806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 371.480650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 371.491517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 371.503950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 371.513824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 371.524112] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 371.533352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 371.541880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 371.551855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 371.562943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 371.574892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 371.585820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 371.596572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 371.610097] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 371.765741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_timeout -r 1 [ 371.780246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_timeout -r 1 [ 371.794574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 371.863946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 371.873793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 371.884384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 371.896329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 371.906953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 371.915563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 371.925799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 371.935952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 371.946726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 371.957748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 371.968322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 371.979364] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 371.990181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 372.001062] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 372.013595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 372.175750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_error -r 1 [ 372.188059] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_error -r 1 [ 372.199953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 372.206451] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 372.208438] LNet: Skipped 2 previous similar messages [ 372.276521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 372.286877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 372.296403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 372.306507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 372.318701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 372.329948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 372.342492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 372.353894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 372.361036] LNet: 56811:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 372.364783] LNet: 56811:0:(api-ni.c:294:recovery_interval_set()) Skipped 2 previous similar messages [ 372.374086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 372.388206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 372.401401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 372.414691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 372.427808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 372.441535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 372.454680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 372.539204] LNet: 56852:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 372.541921] LNetError: 56852:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 372.550271] LNet: Removed LNI 192.168.204.52@tcp [ 372.681449] Key type .llcrypt unregistered [ 372.682602] Key type ._llcrypt unregistered [ 377.373221] Key type ._llcrypt registered [ 377.374125] Key type .llcrypt registered [ 377.787482] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 21:30:13 (1781659813) [ 378.045216] Key type .llcrypt unregistered [ 378.046107] Key type ._llcrypt unregistered [ 379.844641] Key type ._llcrypt registered [ 379.845562] Key type .llcrypt registered [ 379.905862] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 379.911611] alg: No test for adler32 (adler32-zlib) [ 380.705533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 380.714864] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 383.424269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 386.502040] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 386.513469] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 386.520021] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 386.944158] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_interrupt -r 1 [ 386.960475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e local_interrupt -r 1 [ 386.974278] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_interrupt -r 1 [ 386.988514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e local_interrupt -r 1 [ 387.005080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e local_interrupt -r 1 [ 387.021845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e local_interrupt -r 1 [ 387.038193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e local_interrupt -r 1 [ 387.054789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e local_interrupt -r 1 [ 387.072142] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 387.080310] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 387.083819] LNet: 1 local NIs in recovery (showing 1): 192.168.204.52@tcp [ 387.156260] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 387.169343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 387.181744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 387.190685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 387.201102] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 387.212919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 387.225204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 387.237618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 387.245194] LNet: 58314:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 387.253870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 387.265043] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 387.276217] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 387.287980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 387.298725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 387.310135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 387.322049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 387.490728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_dropped -r 1 [ 387.503890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e local_dropped -r 1 [ 387.519295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_dropped -r 1 [ 387.531690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e local_dropped -r 1 [ 387.544387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e local_dropped -r 1 [ 387.557612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e local_dropped -r 1 [ 387.569847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e local_dropped -r 1 [ 387.582181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e local_dropped -r 1 [ 387.596535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 387.602588] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 387.604256] LNet: Skipped 2 previous similar messages [ 387.674387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 387.686821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 387.698380] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 387.709549] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 387.721951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 387.734506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 387.746611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 387.758768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 387.766821] LNet: 58435:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 387.776133] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 387.787533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 387.799768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 387.811069] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 387.822501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 387.836292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 387.849740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 388.011581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_aborted -r 1 [ 388.025233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e local_aborted -r 1 [ 388.040852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_aborted -r 1 [ 388.053651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e local_aborted -r 1 [ 388.066209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e local_aborted -r 1 [ 388.080897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e local_aborted -r 1 [ 388.092791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e local_aborted -r 1 [ 388.105778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e local_aborted -r 1 [ 388.119358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 388.191847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 388.203213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 388.213939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 388.226335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 388.239233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 388.251617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 388.263937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 388.275160] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 388.286623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 388.297703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 388.310103] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 388.319284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 388.330821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 388.343550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 388.357301] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 388.513844] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_no_route -r 1 [ 388.527931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e local_no_route -r 1 [ 388.542083] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_no_route -r 1 [ 388.552752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e local_no_route -r 1 [ 388.563503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e local_no_route -r 1 [ 388.575725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e local_no_route -r 1 [ 388.591015] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e local_no_route -r 1 [ 388.606314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e local_no_route -r 1 [ 388.619575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 388.626294] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 388.629100] LNet: Skipped 6 previous similar messages [ 388.686219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 388.695183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 388.705784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 388.716986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 388.726670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 388.738035] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 388.750476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 388.763970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 388.771728] LNet: 58677:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 388.774082] LNet: 58677:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 388.779719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 388.790914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 388.801724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 388.812275] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 388.823585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 388.834270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 388.848088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 388.999598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_timeout -r 1 [ 389.012076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e local_timeout -r 1 [ 389.024502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_timeout -r 1 [ 389.040634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e local_timeout -r 1 [ 389.052980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e local_timeout -r 1 [ 389.065570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e local_timeout -r 1 [ 389.080450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e local_timeout -r 1 [ 389.092120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e local_timeout -r 1 [ 389.102659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 389.172074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 389.181918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 389.192129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 389.203812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 389.214597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 389.226284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 389.238748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 389.250104] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 389.261294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 389.273655] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 389.285581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 389.300204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 389.312342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 389.325172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 389.338939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 389.500351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_error -r 1 [ 389.514386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e local_error -r 1 [ 389.528597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_error -r 1 [ 389.543325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e local_error -r 1 [ 389.559858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e local_error -r 1 [ 389.575111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e local_error -r 1 [ 389.590843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e local_error -r 1 [ 389.605429] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e local_error -r 1 [ 389.620769] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 389.697692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 389.707786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 389.718293] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 389.728346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 389.738763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 389.749251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 389.760108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 389.772096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 389.782868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 389.794975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 389.806296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 389.820144] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 389.832615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 389.847181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 389.860094] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 391.920966] LNet: 58995:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 391.923883] LNetError: 58995:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 391.927061] LNet: Removed LNI 192.168.204.52@tcp [ 391.932324] LNet: Removed LNI 192.168.204.52@tcp1 [ 392.069508] Key type .llcrypt unregistered [ 392.070528] Key type ._llcrypt unregistered [ 397.191016] Key type ._llcrypt registered [ 397.192400] Key type .llcrypt registered [ 397.619183] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 21:30:33 (1781659833) [ 397.851259] Key type .llcrypt unregistered [ 397.852264] Key type ._llcrypt unregistered [ 399.735211] Key type ._llcrypt registered [ 399.736819] Key type .llcrypt registered [ 399.800609] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 399.806921] alg: No test for adler32 (adler32-zlib) [ 400.601632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 400.611583] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 403.265891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 403.476953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e remote_dropped -r 1 [ 403.491589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e remote_dropped -r 1 [ 403.508073] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 403.516264] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 403.584890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 403.595108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 403.605775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 403.617925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 403.630106] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 403.643504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 403.654502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 403.666583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 403.673163] LNet: 60241:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 403.681154] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 403.694572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 403.706615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 403.718316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 403.731815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 403.745883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 403.760310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 403.937426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e remote_error -r 1 [ 403.955741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e remote_error -r 1 [ 403.973244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 404.056840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 404.067558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 404.078979] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 404.090512] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 404.100587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 404.111763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 404.122406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 404.131967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 404.142063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 404.151100] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 404.160980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 404.173149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 404.185375] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 404.197367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 404.209684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 404.369803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e remote_timeout -r 1 [ 404.385892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e remote_timeout -r 1 [ 404.399825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 404.406584] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 404.408664] LNet: Skipped 1 previous similar message [ 404.477187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 404.487430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 404.498598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 404.510907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 404.521957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 404.533439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 404.544900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 404.556144] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 404.561604] LNet: 60459:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 404.563669] LNet: 60459:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 404.570823] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 404.580882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 404.591911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 404.602968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 404.612077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 404.621885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 404.633614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 404.714691] LNet: 60500:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 404.717083] LNetError: 60500:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 404.724679] LNet: Removed LNI 192.168.204.52@tcp [ 404.850549] Key type .llcrypt unregistered [ 404.851791] Key type ._llcrypt unregistered [ 409.538186] Key type ._llcrypt registered [ 409.539271] Key type .llcrypt registered [ 409.941166] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 21:30:46 (1781659846) [ 410.210347] Key type .llcrypt unregistered [ 410.211422] Key type ._llcrypt unregistered [ 411.920152] Key type ._llcrypt registered [ 411.921084] Key type .llcrypt registered [ 411.978547] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 411.984553] alg: No test for adler32 (adler32-zlib) [ 412.783563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 412.792772] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 415.425175] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 417.909497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 417.919464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 417.926409] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 418.302640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e remote_dropped -r 1 [ 418.314787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e remote_dropped -r 1 [ 418.326359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e remote_dropped -r 1 [ 418.337661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e remote_dropped -r 1 [ 418.348405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e remote_dropped -r 1 [ 418.359494] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e remote_dropped -r 1 [ 418.369971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e remote_dropped -r 1 [ 418.380222] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e remote_dropped -r 1 [ 418.391292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 418.398192] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 418.401841] LNet: 1 peer NIs in recovery (showing 1): 192.168.204.152@tcp [ 418.463219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 418.471867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 418.480656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 418.490064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 418.498931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 418.507782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 418.517411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 418.526877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 418.533406] LNet: 61962:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 418.539294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 418.549563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 418.560446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 418.571957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 418.580684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 418.589872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 418.599644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 418.608992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 418.749629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e remote_error -r 1 [ 418.760926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e remote_error -r 1 [ 418.771387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e remote_error -r 1 [ 418.781652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e remote_error -r 1 [ 418.794164] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e remote_error -r 1 [ 418.804428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e remote_error -r 1 [ 418.815178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e remote_error -r 1 [ 418.828934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e remote_error -r 1 [ 418.841494] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 418.909194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 418.918592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 418.928435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 418.938461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 418.947240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 418.956668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 418.966485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 418.975865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 418.984822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 418.993994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 419.002568] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 419.011440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 419.020662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 419.032913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 419.042944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 419.055418] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 419.201098] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e remote_timeout -r 1 [ 419.211521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e remote_timeout -r 1 [ 419.222347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e remote_timeout -r 1 [ 419.233358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e remote_timeout -r 1 [ 419.244467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e remote_timeout -r 1 [ 419.258032] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e remote_timeout -r 1 [ 419.269930] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e remote_timeout -r 1 [ 419.281801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e remote_timeout -r 1 [ 419.291615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.152@tcp [ 419.297566] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 419.299734] LNet: Skipped 3 previous similar messages [ 419.361170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 419.370931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 419.380326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 419.388956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 419.397881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 419.407239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 419.416259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 419.426088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 419.431444] LNet: 62208:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 419.434643] LNet: 62208:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 419.442550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 419.453614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 419.465177] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 419.477109] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 419.488600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 419.498690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 419.507586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 419.516936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 421.552735] LNet: 62285:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 421.554553] LNetError: 62285:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 421.556790] LNet: Removed LNI 192.168.204.52@tcp [ 421.563242] LNet: Removed LNI 192.168.204.52@tcp1 [ 421.698713] Key type .llcrypt unregistered [ 421.700049] Key type ._llcrypt unregistered [ 426.487892] Key type ._llcrypt registered [ 426.489464] Key type .llcrypt registered [ 426.880717] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 21:31:02 (1781659862) [ 427.331240] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_208 kernel ip2nets deprecated in 2.17.53 [ 427.823517] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 21:31:03 (1781659863) [ 428.064407] Key type .llcrypt unregistered [ 428.065372] Key type ._llcrypt unregistered [ 430.006324] Key type ._llcrypt registered [ 430.007392] Key type .llcrypt registered [ 430.062620] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 430.067937] alg: No test for adler32 (adler32-zlib) [ 430.868414] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 430.877185] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 433.629506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 433.828470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e network_timeout -r 1 [ 433.841391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e network_timeout -r 1 [ 433.855976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 433.864820] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 433.935369] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 433.945611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 433.952078] LNet: 1 local NIs in recovery (showing 1): 192.168.204.52@tcp [ 433.956563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 433.967620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 433.977509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 433.987966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 433.998695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 434.010552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 434.018465] LNet: 63825:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 434.026200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 434.037611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 434.049483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 434.060182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 434.070924] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 434.084181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 434.167784] LNet: 63864:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 434.170377] LNetError: 63864:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 434.180333] LNet: Removed LNI 192.168.204.52@tcp [ 434.311383] Key type .llcrypt unregistered [ 434.312401] Key type ._llcrypt unregistered [ 439.829539] Key type ._llcrypt registered [ 439.831172] Key type .llcrypt registered [ 439.897938] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 439.905275] alg: No test for adler32 (adler32-zlib) [ 440.704866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 440.713318] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 443.529029] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 446.057049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 446.068967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 446.078996] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 446.556649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e network_timeout -r 1 [ 446.573315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e network_timeout -r 1 [ 446.585811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e network_timeout -r 1 [ 446.600329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e network_timeout -r 1 [ 446.617410] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e network_timeout -r 1 [ 446.635170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e network_timeout -r 1 [ 446.653454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e network_timeout -r 1 [ 446.671483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e network_timeout -r 1 [ 446.688658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 446.698958] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 446.787240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 446.802405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 446.817434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 446.829376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 446.840368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 446.850729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 446.860300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 446.871324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 446.877811] LNet: 64738:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 446.881085] LNet: 1 local NIs in recovery (showing 1): 192.168.204.52@tcp [ 446.882453] LNet: 1 peer NIs in recovery (showing 1): 192.168.204.152@tcp [ 446.885878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 446.896665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 446.907767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 446.922768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 446.935071] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 446.948799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 448.922743] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 449.008695] LNet: 64812:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 449.011611] LNetError: 64812:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 449.014708] LNet: Removed LNI 192.168.204.52@tcp [ 449.020287] LNet: Removed LNI 192.168.204.52@tcp1 [ 449.159656] Key type .llcrypt unregistered [ 449.160845] Key type ._llcrypt unregistered [ 454.063835] Key type ._llcrypt registered [ 454.064721] Key type .llcrypt registered [ 454.514220] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 21:31:30 (1781659890) [ 454.628935] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 454.633942] alg: No test for adler32 (adler32-zlib) [ 455.433731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 455.453363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 455.462588] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 455.708279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 455.717954] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 455.999576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 456.025381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 456.041454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 456.053594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 456.065903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 456.077444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 456.090275] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.204.52@tcp [ 456.097562] LNetError: 65909:0:(api-ni.c:4139:lnet_ni_set_healthv()) manually adding local NI 192.168.204.52@tcp to recovery [ 456.480270] LNet: 1 local NIs in recovery (showing 1): 192.168.204.52@tcp [ 456.480371] LNet: There was an unexpected network error while writing to 192.168.204.52: rc = -22 [ 458.528358] LNet: There was an unexpected network error while writing to 192.168.204.52: rc = -22 [ 459.252909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 462.624277] LNet: There was an unexpected network error while writing to 192.168.204.52: rc = -22 [ 463.426596] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 463.438946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 463.459741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 463.466693] LNet: 66027:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 463.469620] LNetError: 66027:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 463.472422] LNet: Removed LNI 192.168.204.52@tcp [ 463.480269] LNet: Removed LNI 192.168.204.52@tcp1 [ 463.487230] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 463.507463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 463.515788] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 463.739302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 463.747647] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 464.031522] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 464.047735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 464.072450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 464.090795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 464.108481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 464.120354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 464.132737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 464.144541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.204.52@tcp [ 464.152964] LNetError: 66299:0:(api-ni.c:4139:lnet_ni_set_healthv()) manually adding local NI 192.168.204.52@tcp to recovery [ 466.592383] LNet: There was an unexpected network error while writing to 192.168.204.52: rc = -22 [ 466.594965] LNet: Skipped 1 previous similar message [ 467.316650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 470.688277] LNet: There was an unexpected network error while writing to 192.168.204.52: rc = -22 [ 475.609066] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 475.621943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 475.632855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 477.582910] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 21:31:53 (1781659913) [ 477.614594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 477.622834] LNet: 67022:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 477.625872] LNetError: 67022:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 477.627560] LNetError: 67022:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 477.629405] LNet: Removed LNI 192.168.204.52@tcp [ 477.635724] LNet: Removed LNI 192.168.204.52@tcp1 [ 477.643378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 477.662325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 477.671175] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 477.889308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 477.896259] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 478.150383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 478.172487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 478.184931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 478.197618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 478.208485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 478.218782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 478.230382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.204.52@tcp --health 0 [ 478.261267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 478.688094] LNet: 1 peer NIs in recovery (showing 1): 192.168.204.52@tcp [ 479.715050] LNet: There was an unexpected network error while writing to 192.168.204.52: rc = -22 [ 479.717321] LNet: Skipped 1 previous similar message [ 493.631210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 493.675817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 493.687042] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 493.697975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.204.52@tcp --health 500 [ 493.729459] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 499.890049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 499.905580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 499.912990] LNet: 67525:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 499.914826] LNetError: 67525:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 499.916771] LNetError: 67525:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 499.918979] LNet: Removed LNI 192.168.204.52@tcp [ 499.924231] LNet: Removed LNI 192.168.204.52@tcp1 [ 499.931206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 499.949595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 499.957760] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 500.216634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 500.224840] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 500.527634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 500.542696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 500.568862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 500.583182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 500.596855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 500.610803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 500.623892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 500.636581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.204.52@tcp --health 0 [ 501.984318] LNet: There was an unexpected network error while writing to 192.168.204.52: rc = -22 [ 501.986547] LNet: Skipped 3 previous similar messages [ 502.766743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 512.089303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 512.099853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 512.109936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 514.062522] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 21:32:30 (1781659950) [ 514.194135] LNet: 68581:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 514.197332] LNetError: 68581:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 514.200393] LNetError: 68581:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 514.202921] LNet: Removed LNI 192.168.204.52@tcp [ 514.208486] LNet: Removed LNI 192.168.204.52@tcp1 [ 514.425980] Key type .llcrypt unregistered [ 514.427557] Key type ._llcrypt unregistered [ 516.450805] Key type ._llcrypt registered [ 516.453235] Key type .llcrypt registered [ 516.512271] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 516.517818] alg: No test for adler32 (adler32-zlib) [ 517.312618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 517.321944] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 520.450042] Lustre: DEBUG MARKER: Initial discovery [ 520.472892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.204.152@tcp [ 521.228867] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag [ 521.251031] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 -e local_error [ 521.267189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.204.152@tcp [ 521.276657] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 521.286064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 521.300660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.204.152@tcp -r 1 -m GET -l 3 [ 521.313149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.204.152@tcp -r 1 -m GET -e local_error [ 521.327786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.204.152@tcp -r 1 -m PUT -l 6 [ 521.440074] LNet: 1 local NIs in recovery (showing 1): 192.168.204.52@tcp [ 521.747594] Lustre: DEBUG MARKER: Force oleg452-client.virtnet to discover 192.168.204.152@tcp (in background) [ 521.765851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.204.152@tcp [ 522.216339] Lustre: DEBUG MARKER: Force oleg452-server to discover 192.168.204.52@tcp [ 522.941908] Lustre: DEBUG MARKER: Wait for 69332 [ 524.832271] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 525.274238] Lustre: DEBUG MARKER: Finished wait on 69332 [ 525.287299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay del -a [ 525.303183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 525.397043] LNet: 69789:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 525.399459] LNetError: 69789:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 525.407378] LNet: Removed LNI 192.168.204.52@tcp [ 525.542668] Key type .llcrypt unregistered [ 525.543501] Key type ._llcrypt unregistered [ 530.700718] Key type ._llcrypt registered [ 530.701847] Key type .llcrypt registered [ 531.154529] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 21:32:47 (1781659967) [ 531.308275] Key type .llcrypt unregistered [ 531.309625] Key type ._llcrypt unregistered [ 532.996044] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 533.058725] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 533.061748] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 533.063771] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 535.248233] Key type ._llcrypt registered [ 535.249228] Key type .llcrypt registered [ 535.312991] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 535.318454] alg: No test for adler32 (adler32-zlib) [ 536.125877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 536.150830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 536.160040] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 536.423812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 536.431222] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 539.214451] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 21:32:55 (1781659975) [ 539.247513] LNet: 71857:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 539.251803] LNetError: 71857:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 539.255492] LNet: Removed LNI 192.168.204.52@tcp [ 539.261838] LNet: Removed LNI 10.1.2.3@tcp [ 539.508410] Key type .llcrypt unregistered [ 539.509862] Key type ._llcrypt unregistered [ 541.576143] Key type ._llcrypt registered [ 541.577136] Key type .llcrypt registered [ 541.637989] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 541.643824] alg: No test for adler32 (adler32-zlib) [ 542.453614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 542.474016] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 542.482885] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 542.714302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 542.724485] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 543.033139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 10.1.2.3@tcp 192.168.204.52@tcp [ 544.096144] LNetError: Unexpected error -22 connecting to 192.168.204.52@tcp at host 192.168.204.52:988 [ 544.950532] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 21:33:01 (1781659981) [ 544.981815] LNet: 72988:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 544.984681] LNetError: 72988:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 544.986975] LNet: Removed LNI 192.168.204.52@tcp [ 544.992313] LNet: Removed LNI 10.1.2.3@tcp [ 545.184287] Key type .llcrypt unregistered [ 545.185497] Key type ._llcrypt unregistered [ 547.032028] Key type ._llcrypt registered [ 547.033208] Key type .llcrypt registered [ 547.090968] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 547.096735] alg: No test for adler32 (adler32-zlib) [ 547.890795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 547.909214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 547.917320] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 548.145495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 548.151542] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 548.388692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 192.168.204.52@tcp1 --nid 192.168.204.52@tcp2 [ 549.569926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.204.52@tcp1 192.168.204.52@tcp2 [ 549.578986] LNetError: 73828:0:(lib-move.c:2007:lnet_handle_find_routed_path()) Unable to send message from 192.168.204.52@tcp1 to 192.168.204.52@tcp2 - Route table may be misconfigured [ 549.588248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.204.52@tcp2 192.168.204.52@tcp1 [ 551.454850] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 21:33:07 (1781659987) [ 551.490412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 551.498513] LNet: 74398:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 551.500588] LNetError: 74398:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 551.502344] LNet: Removed LNI 192.168.204.52@tcp1 [ 552.550571] LNet: Removed LNI 192.168.204.52@tcp2 [ 552.559406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 552.580934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 552.589077] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 552.841703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 552.852404] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 553.150239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 553.164061] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.204.52@tcp 192.168.204.52@tcp [ 553.175797] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.204.52@tcp1 192.168.204.52@tcp1 [ 553.190739] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.204.52@tcp -d 192.168.204.52@tcp -e network_timeout [ 553.206029] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -e network_timeout [ 553.223717] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -e network_timeout [ 553.240500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -e network_timeout [ 553.256217] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 553.263541] LNet: There was an unexpected network error while writing to 192.168.204.52: rc = -22 [ 553.269706] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 553.298575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 553.332850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 553.633095] LNet: 1 local NIs in recovery (showing 1): 192.168.204.52@tcp [ 555.398808] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 21:33:11 (1781659991) [ 555.427902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 555.438034] LNet: 75256:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 555.440805] LNetError: 75256:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 555.443118] LNetError: 75256:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 555.446660] LNet: Removed LNI 192.168.204.52@tcp [ 556.454884] LNet: Removed LNI 192.168.204.52@tcp1 [ 556.462978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 556.492903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 0@lo [ 556.588551] LNet: 75298:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 556.741305] Key type .llcrypt unregistered [ 556.742476] Key type ._llcrypt unregistered [ 560.372086] Key type ._llcrypt registered [ 560.372991] Key type .llcrypt registered [ 560.839894] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 21:33:16 (1781659996) [ 560.968680] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 560.973914] alg: No test for adler32 (adler32-zlib) [ 561.783356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 562.273522] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces [ 562.780615] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 21:33:18 (1781659998) [ 562.819770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 562.830635] LNet: 76389:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 562.842660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 562.872159] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 562.881871] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 563.130804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 563.137890] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 563.414083] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 563.426797] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp1 [ 563.439788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp1 [ 565.386826] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 21:33:21 (1781660001) [ 566.005095] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_220 Need at least 2 remote nodes found oleg452-server [ 566.577323] Lustre: DEBUG MARKER: == sanity-lnet test 221: Add routes w/DD disabled - check aliveness ========================================================== 21:33:22 (1781660002) [ 567.170673] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_221 Need at least 2 remote nodes found oleg452-server [ 567.702238] Lustre: DEBUG MARKER: == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 21:33:23 (1781660003) [ 568.296991] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_222 Need at least 2 remote nodes found oleg452-server [ 568.851630] Lustre: DEBUG MARKER: == sanity-lnet test 223: Check avoid_asym_router_failure=1 w/DD disabled ========================================================== 21:33:24 (1781660004) [ 569.455609] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_223 Need at least 2 remote nodes found oleg452-server [ 569.990790] Lustre: DEBUG MARKER: == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 21:33:26 (1781660006) [ 570.606897] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_224 Need at least 2 remote nodes found oleg452-server [ 571.173193] Lustre: DEBUG MARKER: == sanity-lnet test 225: Check avoid_asym_router_failure=0 w/DD disabled ========================================================== 21:33:27 (1781660007) [ 571.770395] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_225 Need at least 2 remote nodes found oleg452-server [ 572.314590] Lustre: DEBUG MARKER: == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 21:33:28 (1781660008) [ 572.923175] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_226 Need at least 3 remote nodes found oleg452-server [ 573.460841] Lustre: DEBUG MARKER: == sanity-lnet test 227: Check router peer health w/DD disabled ========================================================== 21:33:29 (1781660009) [ 574.037746] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_227 Need at least 3 remote nodes found oleg452-server [ 574.605702] Lustre: DEBUG MARKER: == sanity-lnet test 228: Routes should stay up when health is decremented ========================================================== 21:33:30 (1781660010) [ 575.235308] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_228 Need at least 3 remote nodes found oleg452-server [ 575.808441] Lustre: DEBUG MARKER: == sanity-lnet test 230: Test setting conns-per-peer ===== 21:33:31 (1781660011) [ 575.842336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 575.852410] LNet: 80262:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 575.855216] LNetError: 80262:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 575.859658] LNet: Removed LNI 192.168.204.52@tcp [ 576.870489] LNet: Removed LNI 192.168.204.52@tcp1 [ 576.878516] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 576.899882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 576.908000] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 577.168722] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 4 [ 577.199391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 577.265334] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 577.277398] LNet: 80414:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 577.280929] LNetError: 80414:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 577.284259] LNetError: 80414:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 577.294516] LNet: Removed LNI 192.168.204.52@tcp [ 577.303761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 577.324460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 5 [ 577.337832] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 577.622527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 577.687248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 578.726382] LNet: Removed LNI 192.168.204.52@tcp [ 578.734981] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 578.756836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 578.766502] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 579.031526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 6 [ 579.066839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 579.131203] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 579.139378] LNetError: 80706:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 579.142066] LNetError: 80706:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 579.149428] LNet: Removed LNI 192.168.204.52@tcp [ 579.157522] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 579.176020] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 7 [ 579.186448] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 579.430620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 579.505494] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 579.513679] LNet: 80851:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 579.515813] LNet: 80851:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 580.582383] LNet: Removed LNI 192.168.204.52@tcp [ 580.591804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 580.614347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 580.623765] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 580.903742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 8 [ 580.932260] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 581.004450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 582.054478] LNet: Removed LNI 192.168.204.52@tcp [ 582.062370] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 582.080775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 9 [ 582.088783] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 582.337231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 582.403211] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 582.409853] LNetError: 81144:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 582.412205] LNetError: 81144:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 582.419272] LNet: Removed LNI 192.168.204.52@tcp [ 582.425842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 582.444017] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 582.453078] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 582.671277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 10 [ 582.697590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 582.767244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 583.782322] LNet: Removed LNI 192.168.204.52@tcp [ 583.791817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 583.810209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 11 [ 583.820749] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 584.073373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 584.148284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 584.157984] LNet: 81436:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 584.160058] LNet: 81436:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 585.190325] LNet: Removed LNI 192.168.204.52@tcp [ 585.198408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 585.219973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 585.231203] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 585.484220] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 12 [ 585.514261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 585.593226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 586.662318] LNet: Removed LNI 192.168.204.52@tcp [ 586.670172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 586.691317] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 13 [ 586.700457] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 586.975618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 587.058836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 587.067339] LNetError: 81728:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 587.069423] LNetError: 81728:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 587.076667] LNet: Removed LNI 192.168.204.52@tcp [ 587.083416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 587.101879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 587.109701] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 587.342926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 14 [ 587.369257] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 588.500333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 589.542425] LNet: Removed LNI 192.168.204.52@tcp [ 589.552616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 589.571691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 15 [ 589.580415] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 589.837562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 589.922144] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 590.950289] LNet: Removed LNI 192.168.204.52@tcp [ 590.958345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 590.981556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 590.993909] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 591.241578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 128 [ 591.268212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 591.281872] LNet: Removed LNI 192.168.204.52@tcp [ 591.289374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 591.312099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 591.322540] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 591.588434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer -1 [ 593.820425] Lustre: DEBUG MARKER: == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 21:33:49 (1781660029) [ 593.854977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 593.865298] LNet: 82860:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 593.867341] LNet: 82860:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 593.876133] LNet: Removed LNI 192.168.204.52@tcp [ 593.888502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 593.915431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 [ 593.927446] LNet: Added LNI 192.168.204.52@tcp231 [8/256/0/180] [ 594.329443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 594.352589] LNet: Removed LNI 192.168.204.52@tcp231 [ 594.359717] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 594.386751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 594.395958] LNet: Added LNI 192.168.204.52@tcp231 [8/256/0/180] [ 594.398616] LNet: 83011:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 594.538705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 594.559370] LNet: Removed LNI 192.168.204.52@tcp231 [ 594.565090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=-1 [ 594.574941] LNet: Added LNI 192.168.204.52@tcp231 [8/256/0/180] [ 594.952210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 594.970517] LNet: Removed LNI 192.168.204.52@tcp231 [ 594.979247] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 595.000138] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 595.008448] LNet: Added LNI 192.168.204.52@tcp231 [8/256/0/0] [ 595.011554] LNet: 83163:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 595.149326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 595.169408] LNet: Removed LNI 192.168.204.52@tcp231 [ 595.175302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=0 [ 595.184399] LNet: Added LNI 192.168.204.52@tcp231 [8/256/0/0] [ 595.614719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 595.625088] LNetError: 83306:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 595.628723] LNetError: 83306:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 6 previous similar messages [ 595.638742] LNet: Removed LNI 192.168.204.52@tcp231 [ 595.648071] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 595.677613] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 595.690315] LNet: Added LNI 192.168.204.52@tcp231 [8/256/0/60] [ 595.830182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 595.851431] LNet: Removed LNI 192.168.204.52@tcp231 [ 595.859443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=60 [ 595.871826] LNet: Added LNI 192.168.204.52@tcp231 [8/256/0/60] [ 596.286657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 596.301354] LNet: Removed LNI 192.168.204.52@tcp231 [ 596.309989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 596.337583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 596.349368] LNet: Added LNI 192.168.204.52@tcp231 [8/256/0/180] [ 596.351906] LNet: 83465:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 596.355163] LNet: 83465:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 596.496650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 596.509329] LNet: Removed LNI 192.168.204.52@tcp231 [ 596.517937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=180 [ 596.526935] LNet: Added LNI 192.168.204.52@tcp231 [8/256/0/180] [ 598.992279] Lustre: DEBUG MARKER: == sanity-lnet test 232: Test setting ToS value ========== 21:33:55 (1781660035) [ 599.035810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 599.058259] LNet: Removed LNI 192.168.204.52@tcp231 [ 599.067709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 599.094193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 [ 599.103606] LNet: Added LNI 192.168.204.52@tcp232 [8/256/0/180] [ 599.384268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 599.401205] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 599.422452] LNet: Removed LNI 192.168.204.52@tcp232 [ 599.432150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos -1 [ 599.444380] LNet: Added LNI 192.168.204.52@tcp232 [8/256/0/180] [ 599.731092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 599.745195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 599.763619] LNet: Removed LNI 192.168.204.52@tcp232 [ 599.771327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 104 [ 599.781755] LNet: Added LNI 192.168.204.52@tcp232 [8/256/0/180] [ 600.047710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 600.061170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 600.078592] LNet: Removed LNI 192.168.204.52@tcp232 [ 600.084289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 106 [ 600.094280] LNet: Added LNI 192.168.204.52@tcp232 [8/256/0/180] [ 600.378976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 600.394103] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 600.415451] LNet: Removed LNI 192.168.204.52@tcp232 [ 600.423415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 300 [ 602.335434] Lustre: DEBUG MARKER: == sanity-lnet test 233: Check for successful resends ==== 21:33:58 (1781660038) [ 602.711749] Key type .llcrypt unregistered [ 602.713666] Key type ._llcrypt unregistered [ 604.845600] Key type ._llcrypt registered [ 604.846606] Key type .llcrypt registered [ 604.914780] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 604.921372] alg: No test for adler32 (adler32-zlib) [ 605.725960] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 605.734680] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 608.350523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 610.835331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 610.847682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 610.854380] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 611.344726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_interrupt -r 2 [ 611.360546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e local_interrupt -r 2 [ 611.376935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_interrupt -r 2 [ 611.390807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e local_interrupt -r 2 [ 611.404401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e local_interrupt -r 2 [ 611.419783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e local_interrupt -r 2 [ 611.436376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e local_interrupt -r 2 [ 611.453926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e local_interrupt -r 2 [ 611.469524] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.204.52@tcp 192.168.204.152@tcp [ 611.478242] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 611.480393] LNet: 1 local NIs in recovery (showing 1): 192.168.204.52@tcp [ 611.569545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 611.584531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 611.598282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 611.611043] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 611.622740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 611.633231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 611.644352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 611.654773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 611.661745] LNet: 85961:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 611.668306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 611.679892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 611.690821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 611.702046] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 611.714453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 611.727184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 611.919586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_dropped -r 2 [ 611.937541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e local_dropped -r 2 [ 611.953402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_dropped -r 2 [ 611.968805] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e local_dropped -r 2 [ 611.984458] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e local_dropped -r 2 [ 611.998916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e local_dropped -r 2 [ 612.015188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e local_dropped -r 2 [ 612.033475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e local_dropped -r 2 [ 612.050690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.204.52@tcp 192.168.204.152@tcp [ 612.061837] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 612.148564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 612.158958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 612.173612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 612.188400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 612.203238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 612.218831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 612.230995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 612.242748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 612.248250] LNet: 86081:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 612.254970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 612.268852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 612.279836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 612.291311] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 612.304022] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 612.318060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 612.499213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_aborted -r 2 [ 612.514302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e local_aborted -r 2 [ 612.529731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_aborted -r 2 [ 612.546571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e local_aborted -r 2 [ 612.562662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e local_aborted -r 2 [ 612.577851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e local_aborted -r 2 [ 612.595576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e local_aborted -r 2 [ 612.615186] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e local_aborted -r 2 [ 612.633870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.204.52@tcp 192.168.204.152@tcp [ 612.727676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 612.741174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 612.756473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 612.771796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 612.785242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 612.798729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 612.810503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 612.821482] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 612.832305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 612.843867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 612.855934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 612.869536] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 612.884368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 612.896213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 613.082095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_no_route -r 2 [ 613.098598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e local_no_route -r 2 [ 613.112358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_no_route -r 2 [ 613.124419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e local_no_route -r 2 [ 613.142287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e local_no_route -r 2 [ 613.159457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e local_no_route -r 2 [ 613.172421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e local_no_route -r 2 [ 613.186783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e local_no_route -r 2 [ 613.201442] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.204.52@tcp 192.168.204.152@tcp [ 613.209459] LNet: There was an unexpected network error while writing to 192.168.204.152: rc = -22 [ 613.211443] LNet: Skipped 2 previous similar messages [ 613.293208] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 613.305264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 613.316566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 613.327506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 613.337783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 613.347740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 613.358944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 613.370935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 613.378232] LNet: 86321:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 613.381244] LNet: 86321:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 613.389486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 613.400360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 613.411989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 613.423129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 613.434309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 613.446081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 613.621421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e local_timeout -r 2 [ 613.635971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e local_timeout -r 2 [ 613.650979] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e local_timeout -r 2 [ 613.664418] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e local_timeout -r 2 [ 613.677424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e local_timeout -r 2 [ 613.691032] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e local_timeout -r 2 [ 613.704688] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e local_timeout -r 2 [ 613.717116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e local_timeout -r 2 [ 613.731303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.204.52@tcp 192.168.204.152@tcp [ 613.812748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 613.822358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 613.833469] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 613.844439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 613.854580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 613.867060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 613.879027] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 613.892332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 613.905104] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 613.916750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 613.929886] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 613.940865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 613.953208] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 613.965134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 614.147579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp -m GET -e remote_dropped -r 2 [ 614.163897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.152@tcp1 -m GET -e remote_dropped -r 2 [ 614.176561] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp -m GET -e remote_dropped -r 2 [ 614.190082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp -d 192.168.204.52@tcp1 -m GET -e remote_dropped -r 2 [ 614.203481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp -m GET -e remote_dropped -r 2 [ 614.216091] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.152@tcp1 -m GET -e remote_dropped -r 2 [ 614.231436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp -m GET -e remote_dropped -r 2 [ 614.246683] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.204.52@tcp1 -d 192.168.204.52@tcp1 -m GET -e remote_dropped -r 2 [ 614.263874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.204.52@tcp 192.168.204.152@tcp [ 614.273906] LNet: 1 peer NIs in recovery (showing 1): 192.168.204.152@tcp [ 614.346440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 614.358764] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 614.370871] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 614.384495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 614.395886] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 614.408047] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 614.421782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 614.432985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 614.445374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 614.458552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 614.473309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 614.487283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 614.498917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 614.510591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 616.703318] LNet: 86635:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 616.706592] LNetError: 86635:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 616.709643] LNet: Removed LNI 192.168.204.52@tcp [ 616.718320] LNet: Removed LNI 192.168.204.52@tcp1 [ 616.854469] Key type .llcrypt unregistered [ 616.855621] Key type ._llcrypt unregistered [ 621.977698] Key type ._llcrypt registered [ 621.978693] Key type .llcrypt registered [ 622.387804] Lustre: DEBUG MARKER: == sanity-lnet test 234: Verify NI status changes with manual health value transitions ========================================================== 21:34:18 (1781660058) [ 622.504613] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 622.510779] alg: No test for adler32 (adler32-zlib) [ 623.302740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 623.317668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 623.323949] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 623.582074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 0 [ 623.594558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.204.52@tcp [ 623.602958] LNetError: 87611:0:(api-ni.c:4139:lnet_ni_set_healthv()) manually adding local NI 192.168.204.52@tcp to recovery [ 623.632390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --nid 192.168.204.52@tcp [ 623.664681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 625.483280] Lustre: DEBUG MARKER: == sanity-lnet test 235: Verify NI status DOWN when health reaches 0 after failures ========================================================== 21:34:21 (1781660061) [ 625.514934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 625.523985] LNet: 88196:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 625.526031] LNetError: 88196:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 625.538811] LNet: Removed LNI 192.168.204.52@tcp [ 625.545954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 625.563664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 625.572124] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 625.789699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 625.797291] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 626.090184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 100 --nid 192.168.204.52@tcp [ 626.099295] LNetError: 88465:0:(api-ni.c:4139:lnet_ni_set_healthv()) manually adding local NI 192.168.204.52@tcp to recovery [ 626.592303] LNet: 1 local NIs in recovery (showing 1): 192.168.204.52@tcp [ 626.593058] LNet: There was an unexpected network error while writing to 192.168.204.52: rc = -22 [ 627.171237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 900 --nid 192.168.204.52@tcp [ 631.115831] Lustre: DEBUG MARKER: == sanity-lnet test 236: Local NI state propagates to routers ========================================================== 21:34:27 (1781660067) [ 631.662157] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_236 Need at least 2 remote nodes found oleg452-server [ 632.209505] Lustre: DEBUG MARKER: == sanity-lnet test 237: Check MR forwarding feature ===== 21:34:28 (1781660068) [ 632.818508] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_237 Need at least 2 remote nodes found oleg452-server [ 633.291043] Lustre: DEBUG MARKER: == sanity-lnet test 238: Check MR forwarding feature skipped for non-MR source ========================================================== 21:34:29 (1781660069) [ 633.855518] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_238 Need at least 2 remote nodes found oleg452-server [ 634.328358] Lustre: DEBUG MARKER: == sanity-lnet test 241: Check setting LND timeout value via lnetctl updates tunables ========================================================== 21:34:30 (1781660070) [ 634.358618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 634.367424] LNet: 90090:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 634.370328] LNetError: 90090:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 634.373320] LNet: Removed LNI 192.168.204.52@tcp [ 634.380686] LNet: Removed LNI 192.168.204.52@tcp1 [ 634.388520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 634.408589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 634.417976] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 634.641563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set -h [ 634.664341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 51 [ 634.678169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 634.700266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 0 [ 634.716155] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 634.732135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 50 [ 636.631550] Lustre: DEBUG MARKER: == sanity-lnet test 245: Check read-only params do not appear in export --backup ========================================================== 21:34:32 (1781660072) [ 636.663058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 636.673323] LNet: 90814:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 636.675766] LNetError: 90814:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 636.678026] LNetError: 90814:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 636.685782] LNet: Removed LNI 192.168.204.52@tcp [ 636.694371] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 636.716804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 636.726487] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 638.985020] Lustre: DEBUG MARKER: == sanity-lnet test 250: test that linux routes are added ========================================================== 21:34:35 (1781660075) [ 639.011584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 639.021460] LNet: 91516:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 639.024738] LNetError: 91516:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 639.031299] LNet: Removed LNI 192.168.204.52@tcp [ 639.040394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 639.060924] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 639.070872] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 641.359693] Lustre: DEBUG MARKER: == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 21:34:37 (1781660077) [ 641.812484] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_251 Need kfi NETTYPE [ 642.294804] Lustre: DEBUG MARKER: == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 21:34:38 (1781660078) [ 642.438460] LNet: Removed LNI 192.168.204.52@tcp [ 642.629812] Key type .llcrypt unregistered [ 642.631215] Key type ._llcrypt unregistered [ 644.546600] Key type ._llcrypt registered [ 644.548071] Key type .llcrypt registered [ 644.606707] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 644.612399] alg: No test for adler32 (adler32-zlib) [ 645.405426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 645.413514] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 648.132926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 649.587195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --timeout 15 192.168.204.152@tcp [ 650.742608] LNet: 92971:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 650.744745] LNetError: 92971:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 650.755330] LNet: Removed LNI 192.168.204.52@tcp [ 650.904655] Key type .llcrypt unregistered [ 650.906045] Key type ._llcrypt unregistered [ 654.458224] Key type ._llcrypt registered [ 654.459563] Key type .llcrypt registered [ 654.857925] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253 [ 655.311524] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254 [ 655.781535] Lustre: DEBUG MARKER: == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 21:34:51 (1781660091) [ 656.260436] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_255 kernel ip2nets deprecated in 2.17.53 [ 656.771484] Lustre: DEBUG MARKER: == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 21:34:52 (1781660092) [ 657.308405] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_256 Need at least 2 remote nodes found oleg452-server [ 657.793980] Lustre: DEBUG MARKER: == sanity-lnet test 257: Test graceful router shutdown/startup ========================================================== 21:34:53 (1781660093) [ 658.323717] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_257 Need at least 3 remote nodes found oleg452-server [ 658.812385] Lustre: DEBUG MARKER: == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 21:34:54 (1781660094) [ 659.177531] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 659.228460] alg: No test for adler32 (adler32-zlib) [ 660.095083] Lustre: Lustre: Build Version: 2.17.54_40_g6e23802 [ 660.177292] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 661.752136] Key type lgssc registered [ 662.097613] Lustre: Echo OBD driver; http://www.lustre.org/ [ 669.257507] Key type lgssc unregistered [ 669.372285] LNet: 96140:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 669.374857] LNetError: 96140:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 669.383255] LNet: Removed LNI 192.168.204.52@tcp [ 669.616458] Key type .llcrypt unregistered [ 669.617723] Key type ._llcrypt unregistered [ 674.230563] Key type ._llcrypt registered [ 674.231559] Key type .llcrypt registered [ 674.623154] Lustre: DEBUG MARKER: == sanity-lnet test 265: Import of invalid CPT should fail ========================================================== 21:35:10 (1781660110) [ 674.730720] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 674.735764] alg: No test for adler32 (adler32-zlib) [ 675.532961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 675.554067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 675.569496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 675.588232] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 675.604300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 677.447874] Lustre: DEBUG MARKER: == sanity-lnet test 266: Validate CPT parsing in network sequence ========================================================== 21:35:13 (1781660113) [ 677.500838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 677.510556] LNet: 97551:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 677.519253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 677.536915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 677.544954] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 677.810422] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 677.817976] LNetError: 97695:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 677.831245] LNet: Removed LNI 192.168.204.52@tcp [ 677.837271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 677.852318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 677.860628] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 678.006159] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 678.013762] LNet: 97723:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 678.016660] LNet: 97723:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 678.023261] LNet: Removed LNI 192.168.204.52@tcp [ 678.030977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 678.048547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 678.057313] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 679.993976] Lustre: DEBUG MARKER: == sanity-lnet test 270: Test NID metadata =============== 21:35:16 (1781660116) [ 680.025062] LNet: 98306:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 680.028766] LNetError: 98306:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 680.031328] LNetError: 98306:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 680.039737] LNet: Removed LNI 192.168.204.52@tcp [ 680.259582] Key type .llcrypt unregistered [ 680.260551] Key type ._llcrypt unregistered [ 682.197145] Key type ._llcrypt registered [ 682.198584] Key type .llcrypt registered [ 682.259344] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 682.264853] alg: No test for adler32 (adler32-zlib) [ 683.061334] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 683.084120] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 683.086124] LNet: *** cfs_fail_loc=e003, val=100*** [ 683.316918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 683.323574] Ping reply received with mapping: [ 683.324554] found metadata reply NI[192.168.204.52@tcp] [ 683.325550] 34 cc a8 c0 00 00 02 00 5a 5a 5a 5a 5a 5a 5a 5a [ 683.326809] 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a [ 683.327854] 5a 5a 5a 5a 5a 5a 5a 5a [ 683.368917] LNet: 98709:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 683.371757] LNetError: 98709:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 685.414311] LNet: Removed LNI 192.168.204.52@tcp [ 685.607388] Key type .llcrypt unregistered [ 685.608401] Key type ._llcrypt unregistered [ 688.963040] Key type ._llcrypt registered [ 688.964304] Key type .llcrypt registered [ 689.380992] Lustre: DEBUG MARKER: == sanity-lnet test 280: Don't panic when request_module fails ========================================================== 21:35:25 (1781660125) [ 689.552174] Key type .llcrypt unregistered [ 689.553146] Key type ._llcrypt unregistered [ 691.431626] Key type ._llcrypt registered [ 691.433084] Key type .llcrypt registered [ 691.487071] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 691.491487] alg: No test for adler32 (adler32-zlib) [ 692.342934] LNetError: 99754:0:(api-ni.c:2518:lnet_load_lnd()) Can't load LND tcp, module ksocklnd, rc=256 [ 692.451278] Key type .llcrypt unregistered [ 692.452824] Key type ._llcrypt unregistered [ 693.996765] Key type ._llcrypt registered [ 693.998357] Key type .llcrypt registered [ 694.421449] Lustre: DEBUG MARKER: == sanity-lnet test 290: Check that lnet acceptor is using explicit address binding ========================================================== 21:35:30 (1781660130) [ 694.574307] Key type .llcrypt unregistered [ 694.575352] Key type ._llcrypt unregistered [ 696.284804] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 696.285989] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 698.418212] Key type ._llcrypt registered [ 698.418961] Key type .llcrypt registered [ 698.474224] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 698.478966] alg: No test for adler32 (adler32-zlib) [ 699.275950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 699.296608] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 703.268724] Lustre: DEBUG MARKER: == sanity-lnet test 291: load lnet using custom accept port for bulk tcp connections ========================================================== 21:35:39 (1781660139) [ 703.303933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 703.315617] LNet: 101526:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 703.318138] LNetError: 101526:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 703.331498] LNet: Removed LNI 10.1.2.3@tcp [ 703.343320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 703.604471] Key type .llcrypt unregistered [ 703.605598] Key type ._llcrypt unregistered [ 705.676473] Key type ._llcrypt registered [ 705.677402] Key type .llcrypt registered [ 705.747752] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 705.755692] alg: No test for adler32 (adler32-zlib) [ 706.570813] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 706.597073] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 706.654072] LNet: 101819:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 706.657202] LNetError: 101819:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 706.666396] LNet: Removed LNI 192.168.204.52@tcp [ 706.903501] Key type .llcrypt unregistered [ 706.904693] Key type ._llcrypt unregistered [ 710.773435] Key type ._llcrypt registered [ 710.775196] Key type .llcrypt registered [ 711.229220] Lustre: DEBUG MARKER: == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 21:35:47 (1781660147) [ 711.438470] Key type .llcrypt unregistered [ 711.439817] Key type ._llcrypt unregistered [ 713.681937] Key type ._llcrypt registered [ 713.683387] Key type .llcrypt registered [ 713.742892] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 713.747902] alg: No test for adler32 (adler32-zlib) [ 716.565560] Lustre: DEBUG MARKER: == sanity-lnet test 301: Check for dynamic adds of same/wrong interface (memory leak) ========================================================== 21:35:52 (1781660152) [ 716.596882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 716.614711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 716.645259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 716.657046] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 716.913314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 716.926221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if ens2 [ 716.940479] LNet: Removed LNI 192.168.204.52@tcp [ 717.019388] LNet: 103560:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 717.170267] Key type .llcrypt unregistered [ 717.171527] Key type ._llcrypt unregistered [ 721.000406] Key type ._llcrypt registered [ 721.001341] Key type .llcrypt registered [ 721.442514] Lustre: DEBUG MARKER: == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 21:35:57 (1781660157) [ 721.559396] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 721.564771] alg: No test for adler32 (adler32-zlib) [ 722.364885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 722.387322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 722.400832] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 722.681438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 722.695139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug peer --nid 192.168.204.52@tcp [ 722.701959] LNet: Conn 0 [type, closing, crefcnt, srefcnt]: 2, 0, 1, 1 [ 722.703420] LNet: Conn 0 rx [scheduled, ready, state]: 0, 0, 1 [ 722.704767] LNet: Conn 0 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 722, 0, 0 [ 722.707169] LNet: Conn 0 sched [nconns, cpt]: 4, 0 [ 722.709025] LNet: Conn 1 [type, closing, crefcnt, srefcnt]: 3, 0, 1, 1 [ 722.710509] LNet: Conn 1 rx [scheduled, ready, state]: 0, 0, 1 [ 722.711722] LNet: Conn 1 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 722, 0, 0 [ 722.715099] LNet: Conn 1 sched [nconns, cpt]: 4, 0 [ 722.716483] LNet: Conn 2 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 722.717950] LNet: Conn 2 rx [scheduled, ready, state]: 0, 0, 1 [ 722.719702] LNet: Conn 2 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 722, 0, 772 [ 722.722597] LNet: Conn 2 sched [nconns, cpt]: 4, 0 [ 722.724383] LNet: Conn 3 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 722.725897] LNet: Conn 3 rx [scheduled, ready, state]: 0, 0, 1 [ 722.727453] LNet: Conn 3 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 722, 0, 772 [ 722.731267] LNet: Conn 3 sched [nconns, cpt]: 4, 0 [ 724.846722] Lustre: DEBUG MARKER: == sanity-lnet test 303: Check peer NI health after link down ========================================================== 21:36:00 (1781660160) [ 724.988128] LNet: 105097:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 724.992177] LNetError: 105097:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 725.002559] LNet: Removed LNI 192.168.204.52@tcp [ 725.218465] Key type .llcrypt unregistered [ 725.219628] Key type ._llcrypt unregistered [ 727.260456] Key type ._llcrypt registered [ 727.262015] Key type .llcrypt registered [ 727.331134] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 727.338296] alg: No test for adler32 (adler32-zlib) [ 728.135841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 728.145186] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 730.969872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 733.565838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 733.578969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 733.586400] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 733.893389] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 733.927782] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 733.929438] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 733.931101] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 736.075408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp99 --if test1pg [ 736.086318] LNet: Added LNI 10.1.2.3@tcp99 [8/256/0/180] [ 739.386631] Lustre: DEBUG MARKER: == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 21:36:15 (1781660175) [ 739.436910] LNet: 106596:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 739.440528] LNetError: 106596:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 739.444146] LNet: Removed LNI 192.168.204.52@tcp [ 740.448164] LNet: Removed LNI 192.168.204.52@tcp1 [ 740.449326] LNetError: 106596:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 740.451761] LNetError: 106596:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 740.460658] LNet: Removed LNI 10.1.2.3@tcp99 [ 740.705557] Key type .llcrypt unregistered [ 740.707293] Key type ._llcrypt unregistered [ 742.658771] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 742.661326] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 744.843898] Key type ._llcrypt registered [ 744.845451] Key type .llcrypt registered [ 744.919623] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 744.926265] alg: No test for adler32 (adler32-zlib) [ 745.742258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 745.770803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 745.783438] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 746.073428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 746.081808] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 746.854217] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.1.2.3@tcp --lock_prim [ 746.873420] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 746.892831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 10.1.2.3@tcp --state 1048577 [ 746.903542] LNetError: 107344:0:(peer.c:2175:lnet_del_peer_ni()) peer 10.1.2.3@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 746.918793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 746.935967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 747.079553] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 747.110324] LNet: 107364:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 747.113435] LNetError: 107364:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 747.116562] LNet: Removed LNI 192.168.204.52@tcp [ 747.123398] LNet: Removed LNI 10.1.2.3@tcp [ 747.347335] Key type .llcrypt unregistered [ 747.348441] Key type ._llcrypt unregistered [ 751.024415] Key type ._llcrypt registered [ 751.025437] Key type .llcrypt registered [ 751.485670] Lustre: DEBUG MARKER: == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 21:36:27 (1781660187) [ 751.602918] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 751.607589] alg: No test for adler32 (adler32-zlib) [ 752.409789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 752.430225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 752.437776] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 752.691874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.204.52@tcp [ 752.717226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping oleg452-client@tcp [ 754.769139] Lustre: DEBUG MARKER: == sanity-lnet test 310: Set timeout and verify ========== 21:36:30 (1781660190) [ 754.802301] LNet: 108890:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 754.806781] LNetError: 108890:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 754.817862] LNet: Removed LNI 192.168.204.52@tcp [ 755.040259] Key type .llcrypt unregistered [ 755.041209] Key type ._llcrypt unregistered [ 757.118925] Key type ._llcrypt registered [ 757.120546] Key type .llcrypt registered [ 757.174447] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 757.180375] alg: No test for adler32 (adler32-zlib) [ 757.979841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure --all [ 757.993275] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 758.006668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 760.098969] Lustre: DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 21:36:36 (1781660196) [ 760.597699] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type [ 761.190420] Lustre: DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 21:36:37 (1781660197) [ 761.724858] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type [ 762.259134] Lustre: DEBUG MARKER: == sanity-lnet test 350: Check import --exec of ping/discover ========================================================== 21:36:38 (1781660198) [ 762.295919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 762.307526] LNet: 110308:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 762.310924] LNetError: 110308:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 762.322961] LNet: Removed LNI 192.168.204.52@tcp [ 762.333204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 762.356375] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 762.367283] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 762.640187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 762.657731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 764.910461] Lustre: DEBUG MARKER: == sanity-lnet test 400: Check for udsp add/delete net rule without net num ========================================================== 21:36:40 (1781660200) [ 764.947867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 764.955620] LNet: 111017:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 764.957839] LNetError: 111017:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 764.965186] LNet: Removed LNI 192.168.204.52@tcp [ 764.972783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 764.986425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --priority 0 [ 764.995157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 765.162320] Key type .llcrypt unregistered [ 765.163373] Key type ._llcrypt unregistered [ 768.672879] Key type ._llcrypt registered [ 768.673868] Key type .llcrypt registered [ 769.077339] Lustre: DEBUG MARKER: == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 21:36:45 (1781660205) [ 769.187314] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 769.192896] alg: No test for adler32 (adler32-zlib) [ 769.995420] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 770.018079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 770.028075] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 770.252529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 770.272696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 772.179821] Lustre: DEBUG MARKER: == sanity-lnet test 402: Destination net rule should not panic ========================================================== 21:36:48 (1781660208) [ 772.211552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 772.220097] LNet: 112581:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 772.222652] LNetError: 112581:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 772.230539] LNet: Removed LNI 192.168.204.52@tcp [ 772.241033] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 772.264987] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst kfi --priority 0 [ 772.279192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 402@kfi [ 774.193150] Lustre: DEBUG MARKER: == sanity-lnet test 403: Add and delete udsp net and nid rules ========================================================== 21:36:50 (1781660210) [ 774.223304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 774.233891] LNet: 113160:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 774.245233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 774.273145] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 774.382128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 774.394297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 774.416272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src 1.1.1.1@tcp --prio 1 [ 774.562226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 774.575298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 774.607288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 774.622626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 774.650617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 774.778678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 774.790282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 774.808942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst 1.1.1.1@tcp --prio 1 [ 774.934452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 774.944790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 776.993426] Lustre: DEBUG MARKER: == sanity-lnet test 404: Check udsp src/dst net rule ===== 21:36:53 (1781660213) [ 777.023388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 777.034201] LNet: 113794:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 777.037241] LNet: 113794:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 777.048404] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 777.070875] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 777.079835] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 777.319345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 777.326256] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 777.609535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 777.629530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 777.706020] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 777.714419] LNetError: 114077:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 777.716905] LNet: Removed LNI 192.168.204.52@tcp [ 777.725371] LNet: Removed LNI 192.168.204.52@tcp2 [ 777.733435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 777.755987] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 777.758887] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 777.760795] LNet: 114083:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 777.901974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 777.982119] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 779.040164] LNet: Removed LNI 192.168.204.52@tcp [ 779.041695] LNetError: 114120:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 779.043850] LNetError: 114120:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 779.053318] LNet: Removed LNI 192.168.204.52@tcp2 [ 779.061478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 779.077454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 779.084861] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 779.306723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 779.315932] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 779.586233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 779.600586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 779.619279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 779.691542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 779.699462] LNet: 114404:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 779.701552] LNet: 114404:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 780.704131] LNet: Removed LNI 192.168.204.52@tcp [ 781.734265] LNet: Removed LNI 192.168.204.52@tcp2 [ 781.741372] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 781.760492] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 781.762836] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 781.764863] LNet: 114411:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 781.893530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 781.957565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 781.964611] LNetError: 114448:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 781.966979] LNetError: 114448:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 781.969461] LNet: Removed LNI 192.168.204.52@tcp [ 783.014271] LNet: Removed LNI 192.168.204.52@tcp2 [ 783.021034] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 783.037745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 783.047395] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 783.263182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 783.270597] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 783.527157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 783.545569] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 783.628587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 784.672100] LNet: Removed LNI 192.168.204.52@tcp [ 784.679347] LNet: Removed LNI 192.168.204.52@tcp2 [ 784.686967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 784.707990] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 784.710092] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 784.712458] LNet: 114733:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 784.889757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 784.961614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 784.967748] LNet: 114770:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 784.969873] LNet: 114770:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 786.016075] LNet: Removed LNI 192.168.204.52@tcp [ 786.017728] LNetError: 114770:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 786.020507] LNetError: 114770:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 4 previous similar messages [ 786.028363] LNet: Removed LNI 192.168.204.52@tcp2 [ 786.035718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 786.054945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 786.062619] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 786.279349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 786.287218] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 786.531557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 786.546312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 786.566958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 786.641213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 787.680100] LNet: Removed LNI 192.168.204.52@tcp [ 788.710308] LNet: Removed LNI 192.168.204.52@tcp2 [ 788.716571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 788.733309] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 788.735363] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 788.737118] LNet: 115061:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 788.876238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 788.929150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 788.935899] LNet: Removed LNI 192.168.204.52@tcp [ 789.990295] LNet: Removed LNI 192.168.204.52@tcp2 [ 789.997388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 790.014640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 790.022271] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 790.241686] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 790.249580] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 790.528090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 790.538079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 790.558813] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 790.675267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 791.712133] LNet: Removed LNI 192.168.204.52@tcp [ 791.720352] LNet: Removed LNI 192.168.204.52@tcp2 [ 791.727345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 791.748795] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 791.751102] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 791.886656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 791.984332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 793.056234] LNet: Removed LNI 192.168.204.52@tcp [ 793.064305] LNet: Removed LNI 192.168.204.52@tcp2 [ 793.072117] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 793.091047] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 793.102393] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 793.350860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 793.360318] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 793.624557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 793.638421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 793.648245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 793.665156] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 793.763032] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 793.770455] LNet: 115754:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 793.772627] LNet: 115754:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 4 previous similar messages [ 794.784192] LNet: Removed LNI 192.168.204.52@tcp [ 794.786268] LNetError: 115754:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 794.789913] LNetError: 115754:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 9 previous similar messages [ 794.803425] LNet: Removed LNI 192.168.204.52@tcp2 [ 794.810959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 794.831320] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 794.833342] LNet: Added LNI 192.168.204.52@tcp2 [8/256/0/180] [ 794.835884] LNet: 115760:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 794.839717] LNet: 115760:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 794.991381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 797.151816] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_410 skipping ALWAYS excluded test 410 [ 797.618926] Lustre: DEBUG MARKER: == sanity-lnet test 450: Check import of multiple interfaces ========================================================== 21:37:13 (1781660233) [ 797.647124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 797.656117] LNet: Removed LNI 192.168.204.52@tcp [ 798.694351] LNet: Removed LNI 192.168.204.52@tcp2 [ 798.701484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 798.782464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 798.792504] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 799.002268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 799.010632] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 799.273879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 799.282764] LNet: Removed LNI 192.168.204.52@tcp [ 799.292370] LNet: Removed LNI 10.1.2.3@tcp [ 799.300222] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 799.321622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-450.yaml [ 799.330675] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 799.332302] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 799.476872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del /tmp/sanity-lnet-450.yaml [ 799.484556] LNet: Removed LNI 192.168.204.52@tcp [ 799.494735] LNet: Removed LNI 10.1.2.3@tcp [ 801.417632] Lustre: DEBUG MARKER: == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 21:37:17 (1781660237) [ 801.442409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 801.457436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 801.491614] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 801.538728] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 801.541601] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 801.543284] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 803.668592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 803.678786] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 803.946053] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 803.952066] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 804.606642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.52@tcp [ 804.620051] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d *@tcp -r 1 -l 1 -m PUT [ 804.643037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 192.168.204.52@tcp -m PUT -e local_timeout -r 1 [ 804.664856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 10.1.2.3@tcp -m PUT -e local_timeout -r 1 [ 804.684773] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 804.686134] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 804.765283] LNetError: 117456:0:(lib-move.c:4714:lnet_parse()) 192.168.204.52@tcp, src 192.168.204.52@tcp: Dropping PUT (error -108 looking up sender) [ 806.816172] LNet: Removed LNI 192.168.204.52@tcp [ 806.825256] LNet: Removed LNI 10.1.2.3@tcp [ 806.959058] Key type .llcrypt unregistered [ 806.960975] Key type ._llcrypt unregistered [ 810.379077] Key type ._llcrypt registered [ 810.380090] Key type .llcrypt registered [ 810.770845] Lustre: DEBUG MARKER: == sanity-lnet test 501: Verify lnetctl peer set --health (SR) ========================================================== 21:37:26 (1781660246) [ 810.872686] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 810.878504] alg: No test for adler32 (adler32-zlib) [ 811.676594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 811.784346] LNet: 118765:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 812.001394] Key type .llcrypt unregistered [ 812.002851] Key type ._llcrypt unregistered [ 813.771338] Key type ._llcrypt registered [ 813.772266] Key type .llcrypt registered [ 813.833409] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 813.837914] alg: No test for adler32 (adler32-zlib) [ 814.637790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 814.646200] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 817.340351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 817.394312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 817.595686] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.204.152@tcp [ 817.691321] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 817.696078] LNet: 1 peer NIs in recovery (showing 1): 192.168.204.152@tcp [ 817.706853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 817.721282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 817.735784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 817.748557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 817.762470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 817.776148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 817.789579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 817.796987] LNet: 119213:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 817.805075] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 817.820875] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 817.837391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 817.851519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 817.864511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 817.876429] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 818.112388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 818.204491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 818.220118] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 818.234193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 818.248395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 818.261072] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 818.274543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 818.287596] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 818.300809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 818.308479] LNet: 119330:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 818.315955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 818.328795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 818.341556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 818.355332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 818.368775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 818.382572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 818.619531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.204.152@tcp [ 818.708560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 818.717374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 818.727209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 818.736768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 818.748306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 818.759484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 818.771108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 818.781917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 818.792562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 818.802709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 818.812104] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 818.821286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 818.832182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 818.844429] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 819.039267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 819.109555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 819.120899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 819.131855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 819.142126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 819.151481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 819.162159] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 819.174432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 819.187268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 819.198017] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 819.207816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 819.215955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 819.224646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 819.234820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 819.245789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 819.341434] LNet: 119610:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 819.343424] LNetError: 119610:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 819.351222] LNet: Removed LNI 192.168.204.52@tcp [ 819.462577] Key type .llcrypt unregistered [ 819.463463] Key type ._llcrypt unregistered [ 824.320399] Key type ._llcrypt registered [ 824.321330] Key type .llcrypt registered [ 824.774607] Lustre: DEBUG MARKER: == sanity-lnet test 502: Verify lnetctl peer set --health (MR) ========================================================== 21:37:40 (1781660260) [ 824.896830] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 824.901704] alg: No test for adler32 (adler32-zlib) [ 825.697761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 825.805318] LNet: 120495:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 826.005245] Key type .llcrypt unregistered [ 826.006791] Key type ._llcrypt unregistered [ 827.872298] Key type ._llcrypt registered [ 827.873111] Key type .llcrypt registered [ 827.929022] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 827.934273] alg: No test for adler32 (adler32-zlib) [ 828.739849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 828.752961] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 831.457671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.204.152@tcp [ 833.818765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 833.827730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 833.835183] LNet: Added LNI 192.168.204.52@tcp1 [8/256/0/180] [ 834.109515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 834.277884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.204.152@tcp [ 834.347725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 834.357580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 834.367304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 834.376583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 834.385197] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 834.394765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 834.404285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 834.415305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 834.421487] LNet: 121144:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 834.428690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 834.438060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 834.447678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 834.456432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 834.468046] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 834.478017] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 834.657310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 834.727279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 834.737348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 834.747244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 834.757352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 834.767957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 834.780885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 834.794352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 834.808089] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 834.821252] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 834.833965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 834.847891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 834.858522] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 834.870584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 834.882975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 835.105078] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.204.152@tcp [ 835.180911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 835.189231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 835.198510] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 835.208793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 835.219667] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 835.230317] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 835.239523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 835.250931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 835.258734] LNet: 121378:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 835.261087] LNet: 121378:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 835.269912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 835.284851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 835.298665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 835.311794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 835.324093] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 835.338642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 835.563619] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 835.653036] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 835.667828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 835.679708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 835.693560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 835.706532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 835.718069] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 835.730390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 835.742253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 835.754691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 835.769894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 835.784452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 835.798944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 835.811909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 835.825469] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 837.878226] LNet: 121576:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 837.880976] LNetError: 121576:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 837.883275] LNet: Removed LNI 192.168.204.52@tcp [ 837.890344] LNet: Removed LNI 192.168.204.52@tcp1 [ 838.038901] Key type .llcrypt unregistered [ 838.040237] Key type ._llcrypt unregistered [ 843.011539] Key type ._llcrypt registered [ 843.012378] Key type .llcrypt registered [ 843.433943] Lustre: DEBUG MARKER: == sanity-lnet test 600: lnet_legacy2yaml: basic ip2nets conversion ========================================================== 21:37:59 (1781660279) [ 845.454273] Lustre: DEBUG MARKER: == sanity-lnet test 601: lnet_legacy2yaml: basic routes conversion ========================================================== 21:38:01 (1781660281) [ 847.552547] Lustre: DEBUG MARKER: == sanity-lnet test 602: lnet_legacy2yaml: routes with priority ========================================================== 21:38:03 (1781660283) [ 849.664794] Lustre: DEBUG MARKER: == sanity-lnet test 603: lnet_legacy2yaml: default hop flag ========================================================== 21:38:05 (1781660285) [ 851.741471] Lustre: DEBUG MARKER: == sanity-lnet test 604: lnet_legacy2yaml: default hop with explicit value ========================================================== 21:38:07 (1781660287) [ 853.812200] Lustre: DEBUG MARKER: == sanity-lnet test 605: lnet_legacy2yaml: quote leading star in IP pattern ========================================================== 21:38:09 (1781660289) [ 855.964509] Lustre: DEBUG MARKER: == sanity-lnet test 606: lnet_legacy2yaml: file inputs === 21:38:12 (1781660292) [ 858.077176] Lustre: DEBUG MARKER: == sanity-lnet test 607: lnet_legacy2yaml: options line with combined ip2nets and routes ========================================================== 21:38:14 (1781660294) [ 860.161300] Lustre: DEBUG MARKER: == sanity-lnet test 608: lnet_legacy2yaml: error on invalid rule (missing IP pattern) ========================================================== 21:38:16 (1781660296) [ 862.230761] Lustre: DEBUG MARKER: == sanity-lnet test 609: lnet_legacy2yaml: error on no input ========================================================== 21:38:18 (1781660298) [ 864.198633] Lustre: DEBUG MARKER: == sanity-lnet test 610: lnet_legacy2yaml: multiple files and inline, check order ========================================================== 21:38:20 (1781660300) [ 866.393561] Lustre: DEBUG MARKER: == sanity-lnet test 611: lnet_legacy2yaml: invalid hopcount ========================================================== 21:38:22 (1781660302) [ 868.459716] Lustre: DEBUG MARKER: == sanity-lnet test 612: lnet_legacy2yaml: malformed IP pattern ========================================================== 21:38:24 (1781660304) [ 870.400909] Lustre: DEBUG MARKER: == sanity-lnet test 613: lnet_legacy2yaml: multiple IP patterns in single rule ========================================================== 21:38:26 (1781660306) [ 872.464829] Lustre: DEBUG MARKER: == sanity-lnet test 614: lnet_legacy2yaml: interfaces enumeration with multiple IP ranges ========================================================== 21:38:28 (1781660308) [ 874.594667] Lustre: DEBUG MARKER: == sanity-lnet test 615: lnet_legacy2yaml: modprobe configuration file input ========================================================== 21:38:30 (1781660310) [ 876.705293] Lustre: DEBUG MARKER: == sanity-lnet test 616: lnet_legacy2yaml: combining modprobe file with explicit ip2nets file ========================================================== 21:38:32 (1781660312) [ 878.870451] Lustre: DEBUG MARKER: == sanity-lnet test 617: lnet_legacy2yaml: round-trip tunables from LNet to modprobe to YAML ========================================================== 21:38:34 (1781660314) [ 879.000328] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 879.006066] alg: No test for adler32 (adler32-zlib) [ 879.801583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 879.840968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 879.847608] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 880.214810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 880.222228] LNet: 132268:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 880.225218] LNetError: 132268:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 880.233279] LNet: Removed LNI 192.168.204.52@tcp [ 880.240662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 880.259975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 880.269608] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 882.216237] Lustre: DEBUG MARKER: == sanity-lnet test 618: lnet_legacy2yaml: alternative route format ========================================================== 21:38:38 (1781660318) [ 884.332576] Lustre: DEBUG MARKER: == sanity-lnet test 619: lnet_legacy2yaml: multiple alternative format routes from file ========================================================== 21:38:40 (1781660320) [ 886.343126] Lustre: DEBUG MARKER: == sanity-lnet test 620: lnet_legacy2yaml: mixed legacy and alternative route formats ========================================================== 21:38:42 (1781660322) [ 888.367656] Lustre: DEBUG MARKER: == sanity-lnet test 621: lnet_legacy2yaml: alternative format with invalid hopcount ========================================================== 21:38:44 (1781660324) [ 890.433134] Lustre: DEBUG MARKER: == sanity-lnet test 622: lnet_legacy2yaml: alternative format with default-hop flag ========================================================== 21:38:46 (1781660326) [ 892.487128] Lustre: DEBUG MARKER: == sanity-lnet test 623: lnet_legacy2yaml: basic networks conversion ========================================================== 21:38:48 (1781660328) [ 894.645618] Lustre: DEBUG MARKER: == sanity-lnet test 624: lnet_legacy2yaml: networks with interfaces ========================================================== 21:38:50 (1781660330) [ 896.701093] Lustre: DEBUG MARKER: == sanity-lnet test 625: lnet_legacy2yaml: multiple networks ========================================================== 21:38:52 (1781660332) [ 898.816246] Lustre: DEBUG MARKER: == sanity-lnet test 626: lnet_legacy2yaml: mixed networks, ip2nets, and routes ========================================================== 21:38:54 (1781660334) [ 900.983223] Lustre: DEBUG MARKER: == sanity-lnet test 627: lnet_legacy2yaml: invalid networks format ========================================================== 21:38:57 (1781660337) [ 902.873052] Lustre: DEBUG MARKER: == sanity-lnet test 628: lnet_legacy2yaml: missing delimiter error ========================================================== 21:38:58 (1781660338) [ 904.854971] Lustre: DEBUG MARKER: == sanity-lnet test 629: lnet_legacy2yaml: multiple gateways error ========================================================== 21:39:00 (1781660340) [ 906.784245] Lustre: DEBUG MARKER: == sanity-lnet test complete, duration 806 sec =========== 21:39:02 (1781660342) [ 906.869813] LNet: 139725:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 906.872119] LNetError: 139725:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 906.876644] LNet: Removed LNI 192.168.204.52@tcp [ 907.085500] Key type .llcrypt unregistered [ 907.086876] Key type ._llcrypt unregistered [ 910.777265] Key type ._llcrypt registered [ 910.778185] Key type .llcrypt registered [ 911.013092] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 911.021168] alg: No test for adler32 (adler32-zlib) [ 911.871434] Lustre: Lustre: Build Version: 2.17.54_40_g6e23802 [ 911.944722] LNet: Added LNI 192.168.204.52@tcp [8/256/0/180] [ 913.520125] Key type lgssc registered [ 913.879172] Lustre: Echo OBD driver; http://www.lustre.org/ [ 934.339105] Lustre: Mounted lustre-client [ 936.669844] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 938.756051] Lustre: Unmounted lustre-client [ 950.597642] Key type lgssc unregistered [ 950.718406] LNet: 142638:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 950.721570] LNetError: 142638:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 950.734579] LNet: Removed LNI 192.168.204.52@tcp [ 950.988137] Key type .llcrypt unregistered [ 950.989924] Key type ._llcrypt unregistered