[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 452348832 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54d0-0x000f54df] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829700K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.002000] x2apic enabled [ 0.002007] Switched APIC routing to physical x2apic. [ 0.003000] kvm-guest: setup PV IPIs [ 0.004228] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.005000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.005030] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.006014] pid_max: default: 32768 minimum: 301 [ 0.007157] LSM: Security Framework initializing [ 0.008074] Yama: becoming mindful. [ 0.010020] SELinux: Initializing. [ 0.011000] *** VALIDATE selinux *** [ 0.019558] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025191] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026155] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027114] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028128] *** VALIDATE tmpfs *** [ 0.030323] *** VALIDATE proc *** [ 0.031273] *** VALIDATE cgroup *** [ 0.032012] *** VALIDATE cgroup2 *** [ 0.033296] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035114] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037047] Spectre V2 : User space: Vulnerable [ 0.038014] Speculative Store Bypass: Vulnerable [ 0.040778] debug: unmapping init [mem 0xffffffffbc059000-0xffffffffbc060fff] [ 0.042197] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.043761] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.044026] ... version: 2 [ 0.045017] ... bit width: 48 [ 0.046018] ... generic registers: 4 [ 0.047023] ... value mask: 0000ffffffffffff [ 0.048019] ... max period: 00007fffffffffff [ 0.049017] ... fixed-purpose events: 3 [ 0.050015] ... event mask: 000000070000000f [ 0.051332] rcu: Hierarchical SRCU implementation. [ 0.053557] smp: Bringing up secondary CPUs ... [ 0.054714] x86: Booting SMP configuration: [ 0.055037] .... node #0, CPUs: #1 #2 #3 [ 0.062018] smp: Brought up 1 node, 4 CPUs [ 0.064012] smpboot: Max logical packages: 1 [ 0.065016] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.148757] node 0 deferred pages initialised in 81ms [ 0.150119] devtmpfs: initialized [ 0.151320] x86/mm: Memory block size: 128MB [ 0.155766] gcov: version magic: 0x41383552 [ 0.156703] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.160110] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.162443] pinctrl core: initialized pinctrl subsystem [ 0.165322] [ 0.165709] ************************************************************* [ 0.167011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.169010] ** ** [ 0.170009] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.172011] ** ** [ 0.174013] ** This means that this kernel is built to expose internal ** [ 0.176010] ** IOMMU data structures, which may compromise security on ** [ 0.177013] ** your system. ** [ 0.179011] ** ** [ 0.181010] ** If you see this message and you are not debugging the ** [ 0.182009] ** kernel, report this immediately to your vendor! ** [ 0.184010] ** ** [ 0.185009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.187010] ************************************************************* [ 0.189543] NET: Registered protocol family 16 [ 0.191609] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.194069] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.197083] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.201033] cpuidle: using governor menu [ 0.202820] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.205486] PCI: Using configuration type 1 for base access [ 0.207133] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.216123] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.219034] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.224117] cryptd: max_cpu_qlen set to 1000 [ 0.226265] ACPI: Added _OSI(Module Device) [ 0.228018] ACPI: Added _OSI(Processor Device) [ 0.230014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.231018] ACPI: Added _OSI(Processor Aggregator Device) [ 0.236585] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.243475] ACPI: Interpreter enabled [ 0.245095] ACPI: PM: (supports S0 S3 S4 S5) [ 0.246030] ACPI: Using IOAPIC for interrupt routing [ 0.248168] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.250501] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.259578] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.262062] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.265024] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.268085] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.273320] acpiphp: Slot [2] registered [ 0.274132] acpiphp: Slot [5] registered [ 0.275097] acpiphp: Slot [6] registered [ 0.277091] acpiphp: Slot [3] registered [ 0.278101] acpiphp: Slot [4] registered [ 0.279098] acpiphp: Slot [7] registered [ 0.280095] acpiphp: Slot [8] registered [ 0.281081] acpiphp: Slot [9] registered [ 0.282121] acpiphp: Slot [10] registered [ 0.283126] acpiphp: Slot [11] registered [ 0.284089] acpiphp: Slot [12] registered [ 0.285127] acpiphp: Slot [13] registered [ 0.287110] acpiphp: Slot [14] registered [ 0.288103] acpiphp: Slot [15] registered [ 0.289153] acpiphp: Slot [16] registered [ 0.290081] acpiphp: Slot [17] registered [ 0.291101] acpiphp: Slot [18] registered [ 0.293150] acpiphp: Slot [19] registered [ 0.294103] acpiphp: Slot [20] registered [ 0.295085] acpiphp: Slot [21] registered [ 0.295839] acpiphp: Slot [22] registered [ 0.297056] acpiphp: Slot [23] registered [ 0.297735] acpiphp: Slot [24] registered [ 0.298079] acpiphp: Slot [25] registered [ 0.300178] acpiphp: Slot [26] registered [ 0.301094] acpiphp: Slot [27] registered [ 0.302082] acpiphp: Slot [28] registered [ 0.302954] acpiphp: Slot [29] registered [ 0.304053] acpiphp: Slot [30] registered [ 0.305078] acpiphp: Slot [31] registered [ 0.306088] PCI host bridge to bus 0000:00 [ 0.308019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.310028] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.312027] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.314092] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.316023] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.318028] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.320181] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.323073] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.326169] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.333016] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.337010] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.340020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.342017] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.344016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.346669] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.350055] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.352057] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.355821] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.361017] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.372017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.376040] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.382411] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.388017] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.393027] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.407018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.417067] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.424025] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.432022] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.447024] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.457533] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.459338] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.461263] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.462272] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.464205] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.468110] iommu: Default domain type: Passthrough [ 0.470395] SCSI subsystem initialized [ 0.471112] ACPI: bus type USB registered [ 0.472119] usbcore: registered new interface driver usbfs [ 0.474088] usbcore: registered new interface driver hub [ 0.475063] usbcore: registered new device driver usb [ 0.476138] pps_core: LinuxPPS API ver. 1 registered [ 0.477012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.480089] PTP clock support registered [ 0.482137] EDAC MC: Ver: 3.0.0 [ 0.484149] PCI: Using ACPI for IRQ routing [ 0.485829] NetLabel: Initializing [ 0.488015] NetLabel: domain hash size = 128 [ 0.489012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.492085] NetLabel: unlabeled traffic allowed by default [ 0.494138] vgaarb: loaded [ 0.497219] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.499013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.505429] clocksource: Switched to clocksource kvm-clock [ 0.614642] VFS: Disk quotas dquot_6.6.0 [ 0.616169] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.618752] *** VALIDATE ramfs *** [ 0.619990] *** VALIDATE hugetlbfs *** [ 0.621556] pnp: PnP ACPI init [ 0.623886] pnp: PnP ACPI: found 6 devices [ 0.640232] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.643085] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.644823] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.647053] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.649652] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.652042] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.654658] NET: Registered protocol family 2 [ 0.656907] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.662679] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.666184] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.670658] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.674128] TCP: Hash tables configured (established 65536 bind 65536) [ 0.677157] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.681197] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.684269] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.686741] NET: Registered protocol family 1 [ 0.688973] RPC: Registered named UNIX socket transport module. [ 0.691043] RPC: Registered udp transport module. [ 0.692793] RPC: Registered tcp transport module. [ 0.694537] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.697040] NET: Registered protocol family 44 [ 0.698944] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.701174] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.703846] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.706178] PCI: CLS 0 bytes, default 64 [ 0.707857] Unpacking initramfs... [ 2.110888] debug: unmapping init [mem 0xffff8a06bcc64000-0xffff8a06bffcffff] [ 2.115110] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.117253] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.119972] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 2.619502] Initialise system trusted keyrings [ 2.621056] Key type blacklist registered [ 2.622786] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.632067] zbud: loaded [ 2.634970] *** VALIDATE nfs *** [ 2.635975] *** VALIDATE nfs4 *** [ 2.637016] pstore: using deflate compression [ 2.639478] Platform Keyring initialized [ 2.741224] NET: Registered protocol family 38 [ 2.743319] Key type asymmetric registered [ 2.744864] Asymmetric key parser 'x509' registered [ 2.746912] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.750184] io scheduler mq-deadline registered [ 2.751787] io scheduler kyber registered [ 2.753632] io scheduler bfq registered [ 2.755564] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.758691] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.761799] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.764506] ACPI: Power Button [PWRF] [ 2.769913] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.776834] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.808270] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.838958] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.868247] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.874253] Non-volatile memory driver v1.3 [ 2.876225] Linux agpgart interface v0.103 [ 2.914865] virtio_blk virtio1: [vda] 147160 512-byte logical blocks (75.3 MB/71.9 MiB) [ 2.917961] vda: detected capacity change from 0 to 75345920 [ 2.945102] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.947846] vdb: detected capacity change from 0 to 1073741824 [ 2.963582] libphy: Fixed MDIO Bus: probed [ 2.972761] usbcore: registered new interface driver usbserial_generic [ 2.975867] usbserial: USB Serial support registered for generic [ 2.978639] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.981957] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.983479] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.985457] mousedev: PS/2 mouse device common for all mice [ 2.987662] rtc_cmos 00:05: RTC can wake from S4 [ 2.989898] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.991119] rtc_cmos 00:05: registered as rtc0 [ 2.995037] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.995239] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.998139] intel_pstate: CPU model not supported [ 3.002220] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.006545] hid: raw HID events driver (C) Jiri Kosina [ 3.009239] usbcore: registered new interface driver usbhid [ 3.011497] usbhid: USB HID core driver [ 3.013308] drop_monitor: Initializing network drop monitor service [ 3.015942] Initializing XFRM netlink socket [ 3.018207] NET: Registered protocol family 10 [ 3.022434] Segment Routing with IPv6 [ 3.024300] NET: Registered protocol family 17 [ 3.026374] mpls_gso: MPLS GSO support [ 3.032605] RAS: Correctable Errors collector initialized. [ 3.034914] AVX version of gcm_enc/dec engaged. [ 3.036713] AES CTR mode by8 optimization enabled [ 3.119630] sched_clock: Marking stable (3119587125, 0)->(3984859544, -865272419) [ 3.123689] registered taskstats version 1 [ 3.125974] Loading compiled-in X.509 certificates [ 3.128520] zswap: loaded using pool lzo/zbud [ 3.158450] Key type big_key registered [ 3.169791] Key type encrypted registered [ 3.171616] ima: No TPM chip found, activating TPM-bypass! [ 3.173830] ima: Allocated hash algorithm: sha1 [ 3.175700] ima: No architecture policies found [ 3.177562] evm: Initialising EVM extended attributes: [ 3.179432] evm: security.selinux [ 3.180697] evm: security.ima [ 3.181801] evm: security.capability [ 3.183028] evm: HMAC attrs: 0x1 [ 3.185795] rtc_cmos 00:05: setting system clock to 2026-09-11 12:18:26 UTC (1789129106) [ 3.192424] debug: unmapping init [mem 0xffffffffbd003000-0xffffffffbd1fffff] [ 3.195614] debug: unmapping init [mem 0xffffffffbbd82000-0xffffffffbc058fff] [ 3.202099] Write protecting the kernel read-only data: 28672k [ 3.205676] debug: unmapping init [mem 0xffffffffba403000-0xffffffffba5fffff] [ 3.208816] debug: unmapping init [mem 0xffffffffbad14000-0xffffffffbadfffff] [ 3.245760] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.255270] systemd[1]: Detected virtualization kvm. [ 3.257186] systemd[1]: Detected architecture x86-64. [ 3.259060] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.286248] systemd[1]: No hostname configured. [ 3.288011] systemd[1]: Set hostname to . [ 3.289885] random: systemd: uninitialized urandom read (16 bytes read) [ 3.292653] systemd[1]: Initializing machine ID from random generator. [ 3.438716] random: systemd: uninitialized urandom read (16 bytes read) [ 3.441992] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.446738] random: systemd: uninitialized urandom read (16 bytes read) [ 3.449632] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.453321] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Reached target Local File Systems. [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket. Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Reached target Sockets. [ OK ] Reached target Initrd Root Device. Starting Setup Virtual Console... Starting Create Volatile Files and Directories... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.076155] device-mapper: uevent: version 1.0.3 [ 4.077825] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.799049] virtio_net virtio0 ens2: renamed from eth0 [ 4.809406] scsi host0: ata_piix [ 4.833880] scsi host1: ata_piix [ 4.835225] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 4.836930] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 8.600580] dracut-initqueue[579]: RTNETLINK answers: File exists[ 8.602530] random: fast init done [ 9.689467] random: crng init done [ 9.691020] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.136548] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.373846] printk: systemd: 24 output lines suppressed due to ratelimiting [ 11.622571] SELinux: Disabled at runtime. [ 11.686338] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.696402] systemd[1]: Detected virtualization kvm. [ 11.698361] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.278540] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.281408] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.285562] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.290730] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.294209] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.308573] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.321515] systemd[1]: Activating swap /dev/disk/by-label/SWAP... Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-serial\x2dgetty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Remount Root and Kernel File Systems... [ 12.370893] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice User and Session Slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting POSIX Message Queue File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Mounting Huge Pages File System... [ OK ] Created slice system-getty.slice. [ OK ] Reached target Slices. Starting Apply Kernel Variables... [ OK ] Reached target rpc_pipefs.target. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target Paths. [ OK ] Stopped target Initrd Root File System. Mounting Kernel Debug File System... [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 12.905280] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.210031] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.261960] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.381312] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.397703] EDAC sbridge: Ver: 1.1.2 [ 14.730247] Key type dns_resolver registered [ 15.025792] NFS: Registering the id_resolver key type [ 15.027722] Key type id_resolver registered [ 15.029308] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. Starting Network Manager... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg105-client login: [ 69.551032] hrtimer: interrupt took 3022749 ns [ 80.914661] libcfs: loading out-of-tree module taints kernel. [ 80.988304] Key type ._llcrypt registered [ 80.996466] Key type .llcrypt registered [ 82.002451] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 82.017994] alg: No test for adler32 (adler32-zlib) [ 83.662886] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 84.321030] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 86.072470] Key type lgssc registered [ 88.888347] Lustre: Echo OBD driver; http://www.lustre.org/ [ 248.658111] Lustre: Mounted lustre-client - version 2.17.57_133_g066c38c [ 254.931742] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 273.158979] Lustre: DEBUG MARKER: oleg105-client.virtnet: executing check_logdir /tmp/testlogs/ [ 274.400337] Lustre: lustre-OST0000-osc-ffff8a070a314000: disconnect after 23s idle [ 278.923565] Lustre: DEBUG MARKER: oleg105-client.virtnet: executing yml_node [ 285.095901] Lustre: DEBUG MARKER: Client: 2.17.57.133 [ 288.405800] Lustre: DEBUG MARKER: MDS: 2.17.57.133 [ 291.956563] Lustre: DEBUG MARKER: OSS: 2.17.57.133 [ 293.556903] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Fri Sep 11 08:23:15 EDT 2026 [ 311.278736] Lustre: Unmounted lustre-client [ 342.000080] Key type lgssc unregistered [ 342.381181] LNet: 8434:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 342.396345] LNetError: 8434:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 342.415417] LNet: Removed LNI 192.168.201.5@tcp [ 343.405178] Key type .llcrypt unregistered [ 343.406950] Key type ._llcrypt unregistered [ 374.939628] Key type ._llcrypt registered [ 374.941642] Key type .llcrypt registered [ 375.280474] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 375.320857] alg: No test for adler32 (adler32-zlib) [ 376.282110] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 376.346155] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 376.368459] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show [ 381.820466] Lustre: DEBUG MARKER: excepting tests: 253 254 410 [ 381.963860] LNet: 9324:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 381.983752] LNetError: 9324:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 382.006342] LNet: Removed LNI 192.168.201.5@tcp [ 382.757708] Key type .llcrypt unregistered [ 382.764104] Key type ._llcrypt unregistered [ 390.372385] Key type ._llcrypt registered [ 390.379987] Key type .llcrypt registered [ 391.786523] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 08:24:53 (1789129493) [ 392.530815] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 392.549236] alg: No test for adler32 (adler32-zlib) [ 393.535986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 393.746925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 393.787550] LNet: 9747:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 393.827545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 393.902464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 393.920387] LNet: 9754:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 405.385987] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 08:25:07 (1789129507) [ 405.980407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 406.007350] LNet: 10334:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 406.021736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 406.132366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp [ 415.075208] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 08:25:17 (1789129517) [ 415.169554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 415.233802] LNet: 10911:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 415.383512] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 415.562399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 2.2.2.2@o2ib [ 424.337511] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 08:25:26 (1789129526) [ 424.545842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 424.567500] LNet: 11504:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 424.583112] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 424.631963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib [ 433.590657] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 08:25:34 (1789129534) [ 433.651877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 433.685121] LNet: 12083:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 433.717944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 433.878168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp [ 434.211855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp [ 442.244912] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 08:25:43 (1789129543) [ 442.553737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 442.606726] LNet: 12681:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 442.638358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 442.774980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 443.358738] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 444.033626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib [ 452.767855] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 08:25:54 (1789129554) [ 453.108831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 453.159566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 453.232556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi,[6-12/4]@bxi3f [ 462.490427] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 08:26:04 (1789129564) [ 462.649734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 462.690973] LNet: 13857:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 462.698532] LNet: 13857:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 462.730480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 462.880263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp [ 462.922204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 463.511529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib [ 463.570105] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 464.293970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp [ 464.334130] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 464.901266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib [ 464.935806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 465.699062] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib [ 465.763665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 466.461292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni [ 466.526420] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 467.285660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni [ 467.346304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 467.966037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi [ 468.021730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 468.699682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi [ 468.763745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 469.445069] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f [ 469.520882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 470.136396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f --nid [8-12]@bxi3f [ 470.221160] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 470.971262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi,[3-7]@bxi3f [ 471.079378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 471.781698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 471.861565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 479.128101] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 08:26:20 (1789129580) [ 479.232320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 479.306353] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 479.459157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp [ 479.499242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp [ 488.023828] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 08:26:29 (1789129589) [ 488.268772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 488.361153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 488.505775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 488.559049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 497.571352] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 08:26:39 (1789129599) [ 497.727575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 497.760248] LNet: 15714:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 497.767580] LNet: 15714:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 497.808539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 497.979181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp [ 498.011695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp [ 506.014571] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 08:26:47 (1789129607) [ 506.202425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 506.268088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 506.423923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 506.462114] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 513.709586] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 08:26:55 (1789129615) [ 514.239693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 514.313560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 514.468081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp [ 514.505696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp [ 522.322658] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 08:27:03 (1789129623) [ 522.531879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 522.618310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 522.856447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 522.894948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 532.555282] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 08:27:13 (1789129633) [ 533.179178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 533.239989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 533.446666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 534.326674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 544.232886] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 08:27:25 (1789129645) [ 544.504035] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 544.631937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 545.019246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 545.251719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 553.778487] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 08:27:35 (1789129655) [ 554.050498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 554.111102] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 554.258858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 554.484421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 561.946885] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 08:27:43 (1789129663) [ 562.418624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 562.460893] LNet: 19897:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 562.473665] LNet: 19897:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 6 previous similar messages [ 562.522995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 563.340811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 563.986135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 572.350125] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 08:27:53 (1789129673) [ 572.520684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 572.617461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 572.778172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 572.831559] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 582.360601] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 08:28:03 (1789129683) [ 582.525051] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 582.673969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 582.837679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 582.964479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 591.092537] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 08:28:13 (1789129693) [ 591.500568] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 591.605453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 591.729115] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 19@gni [ 601.466492] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 08:28:23 (1789129703) [ 601.588250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 601.631839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 601.768362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib [ 610.597653] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 08:28:32 (1789129712) [ 610.767389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 610.854684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 611.012516] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni [ 611.760921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni [ 620.500216] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 08:28:41 (1789129721) [ 620.622195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 620.714287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 620.852564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni [ 620.881823] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 22@gni --nid 26@gni [ 629.772047] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 08:28:51 (1789129731) [ 629.887672] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 629.998597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 630.159783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni [ 630.198798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni [ 638.883601] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 08:29:00 (1789129740) [ 639.030126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 639.160930] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 639.340151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni [ 639.395334] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp [ 647.188567] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 08:29:09 (1789129749) [ 647.541726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 647.576543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 647.669807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 647.727597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 655.380139] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 08:29:17 (1789129757) [ 655.616166] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 655.701721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 655.869046] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim [ 656.022879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp [ 656.064943] LNetError: 25871:0:(peer.c:2362:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 656.112319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --force [ 656.178639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show --nid 1.1.1.1@tcp [ 663.589673] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 08:29:24 (1789129764) [ 663.710484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 663.751159] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 663.850692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.zl56sdAdUk [ 663.952710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.zl56sdAdUk [ 664.006677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.zl56sdAdUk [ 672.270229] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 08:29:34 (1789129774) [ 672.658729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 672.711245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 672.782170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp [ 672.827844] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp [ 672.919622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer list [ 680.558877] Lustre: DEBUG MARKER: == sanity-lnet test 40: LNetAddPeer merges a pre-existing non-primary peer NID ========================================================== 08:29:42 (1789129782) [ 680.968147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 681.037439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 681.154284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 681.193534] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 683.115731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 684.850674] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 686.944524] Key type lgssc registered [ 688.528533] Lustre: Echo OBD driver; http://www.lustre.org/ [ 692.965028] LNetError: 27633:0:(lib-move.c:2158:lnet_handle_find_routed_path()) peer 192.168.201.5@tcp99 has no available nets [ 695.804339] Key type lgssc unregistered [ 696.149741] LNet: 28747:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 696.157527] LNet: 28747:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 13 previous similar messages [ 696.176589] LNetError: 28747:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 697.193089] LNet: Removed LNI 192.168.201.5@tcp [ 698.152528] Key type .llcrypt unregistered [ 698.159827] Key type ._llcrypt unregistered [ 706.999805] Key type ._llcrypt registered [ 707.011924] Key type .llcrypt registered [ 708.563482] Lustre: DEBUG MARKER: == sanity-lnet test 50: Enable/disable routing via yaml import ========================================================== 08:30:10 (1789129810) [ 709.173733] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 709.209057] alg: No test for adler32 (adler32-zlib) [ 710.244872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 710.350682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 714.776669] LNet: Message forwarding will be enabled in 2s [ 717.856145] LNet: Message forwarding enabled [ 718.393584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 718.433324] LNet: Message forwarding will stop in 2s to 6s [ 723.279555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 723.897076] LNet: Message forwarding disabled [ 725.963324] LNet: Message forwarding will be enabled in 2s [ 729.056808] LNet: Message forwarding enabled [ 734.692353] Lustre: DEBUG MARKER: == sanity-lnet test 51: Set numa range via yaml import === 08:30:36 (1789129836) [ 735.009575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 735.031775] LNet: 30160:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 736.764210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 736.868728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-51.yaml [ 744.500578] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 08:30:45 (1789129845) [ 744.629716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 744.647846] LNet: 30735:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 744.669383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 744.731481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid foobar [ 744.769527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid foobar [ 744.789742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@o2ib [ 744.811831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --nid 1.1.1.1@tcp [ 744.850745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --nid 1.1.1.1@tcp [ 744.878896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add [ 744.914950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 744.987724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp [ 745.079198] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@foo [ 745.168713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@tcp [ 745.253947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@o2ib [ 745.306754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@gni [ 745.383286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni [ 745.446054] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni [ 745.500903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp [ 745.530300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp [ 745.561113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp [ 745.599258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 745.642969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp --gateway 1@gni [ 745.661559] LNetError: 30785:0:(router.c:625:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni [ 753.743934] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 08:30:55 (1789129855) [ 753.910312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 753.943885] LNet: 31349:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 753.985961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 754.132602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 763.323938] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 08:31:04 (1789129864) [ 766.553360] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_100 Need at least 2 remote nodes found oleg105-server [ 769.427863] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 08:31:10 (1789129870) [ 769.738360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 769.795118] LNet: 32288:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 769.838086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 769.969698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 770.007538] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 770.894507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp1 --gateway 192.168.201.[6-10]@tcp [ 770.919314] LNet: 32422:0:(router.c:729:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 770.980755] LNetError: 32292:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.6@tcp is being used as a gateway but routing feature is not turned on [ 771.005485] LNetError: 32292:0:(lib-lnet.h:1280:lnet_set_route_aliveness()) route to tcp1 through 192.168.201.6@tcp has gone from up to down [ 772.005221] LNetError: 32292:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.8@tcp is being used as a gateway but routing feature is not turned on [ 772.014795] LNetError: 32292:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 9 previous similar messages [ 773.044888] LNetError: 32292:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.8@tcp is being used as a gateway but routing feature is not turned on [ 773.064926] LNetError: 32292:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 4 previous similar messages [ 775.089838] LNetError: 32292:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.6@tcp is being used as a gateway but routing feature is not turned on [ 775.125535] LNetError: 32292:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 9 previous similar messages [ 779.196329] LNetError: 32292:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.7@tcp is being used as a gateway but routing feature is not turned on [ 779.227129] LNetError: 32292:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 19 previous similar messages [ 781.417547] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 08:31:22 (1789129882) [ 784.768368] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_102 Need at least 2 remote nodes found oleg105-server [ 786.453390] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 08:31:28 (1789129888) [ 786.847078] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 786.869559] LNet: 33359:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 786.886897] LNetError: 33359:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 786.939445] LNet: Removed LNI 192.168.201.5@tcp [ 786.986294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 787.081557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 787.145359] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 787.886262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp103 --gateway 192.168.201.[6-11/2]@tcp [ 787.914994] LNet: 33476:0:(router.c:729:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 787.933464] LNet: 33476:0:(router.c:729:lnet_add_route()) Skipped 4 previous similar messages [ 787.973574] LNetError: 33363:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.6@tcp is being used as a gateway but routing feature is not turned on [ 787.992096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route del --net tcp103 --gateway 192.168.201.[6-11/2]@tcp [ 787.995148] LNetError: 33363:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 39 previous similar messages [ 787.995175] LNetError: 33363:0:(lib-lnet.h:1280:lnet_set_route_aliveness()) route to tcp103 through 192.168.201.6@tcp has gone from up to down [ 787.995179] LNetError: 33363:0:(lib-lnet.h:1280:lnet_set_route_aliveness()) Skipped 4 previous similar messages [ 797.436684] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 08:31:38 (1789129898) [ 797.578898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 797.620178] LNet: 34062:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 797.636994] LNetError: 34062:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 797.677558] LNet: Removed LNI 192.168.201.5@tcp [ 797.816486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 797.922587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking -1 [ 797.996524] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 798.068413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 798.197859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 798.316375] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 798.391699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 798.500571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 0 [ 798.640715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 798.714417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 798.822023] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 798.910970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 798.970516] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 799.070771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 1 [ 799.228258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 799.312445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 799.404440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 799.480875] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 799.508414] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 799.562541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 2 [ 799.634100] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 799.661141] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 799.748953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 799.869744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 799.900169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 799.964917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 800.055634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 800.097411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 800.205746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 800.337018] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 800.386425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 800.436894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 4 [ 800.492985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 800.532632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 800.587864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 809.250239] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 08:31:50 (1789129910) [ 811.844562] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_105 Need at least 2 remote nodes found oleg105-server [ 814.101701] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 08:31:55 (1789129915) [ 816.403963] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_106 Need at least 2 remote nodes found oleg105-server [ 818.502334] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 08:32:00 (1789129920) [ 818.940938] LNet: 35470:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 818.944968] LNet: 35470:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 11 previous similar messages [ 819.928110] Key type .llcrypt unregistered [ 819.934244] Key type ._llcrypt unregistered [ 829.367671] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 829.724523] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 829.732651] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 829.754564] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 832.440795] Key type ._llcrypt registered [ 832.462908] Key type .llcrypt registered [ 832.595184] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 832.604555] alg: No test for adler32 (adler32-zlib) [ 833.536774] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 833.598755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 833.629527] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 834.222977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 834.236749] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 836.449537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 836.481122] LNet: Removed LNI 10.1.2.3@tcp [ 836.609362] LNet: 36197:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 836.615622] LNetError: 36197:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 836.643720] LNet: Removed LNI 192.168.201.5@tcp [ 837.483253] Key type .llcrypt unregistered [ 837.484773] Key type ._llcrypt unregistered [ 855.684260] Key type ._llcrypt registered [ 855.692200] Key type .llcrypt registered [ 857.299733] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 08:32:38 (1789129958) [ 857.811390] Key type .llcrypt unregistered [ 857.813524] Key type ._llcrypt unregistered [ 866.210152] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 866.392436] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 866.398227] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 866.405193] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 869.116553] Key type ._llcrypt registered [ 869.118202] Key type .llcrypt registered [ 869.366382] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 869.392185] alg: No test for adler32 (adler32-zlib) [ 870.416501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 870.518426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 870.570631] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 871.469968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 871.505183] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 874.129952] LNet: 37698:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 874.136196] LNetError: 37698:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 874.151336] LNet: Removed LNI 10.1.2.3@tcp [ 874.174100] LNet: Removed LNI 192.168.201.5@tcp [ 874.782618] Key type .llcrypt unregistered [ 874.784399] Key type ._llcrypt unregistered [ 888.046423] Key type ._llcrypt registered [ 888.048132] Key type .llcrypt registered [ 889.599080] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 08:33:11 (1789129991) [ 890.164706] Key type .llcrypt unregistered [ 890.166887] Key type ._llcrypt unregistered [ 897.907306] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 898.174592] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 898.188344] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 898.202846] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 901.203750] Key type ._llcrypt registered [ 901.206735] Key type .llcrypt registered [ 901.473939] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 901.511486] alg: No test for adler32 (adler32-zlib) [ 902.490675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 902.564471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 902.624333] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 905.240565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 905.255838] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 907.078304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 907.099877] LNet: Removed LNI 10.1.2.3@tcp [ 907.114396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg:0 [ 907.141947] LNet: Removed LNI 10.1.2.31@tcp [ 907.173340] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 907.216951] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 908.011477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 908.043199] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 909.451196] LNet: 39644:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 909.471748] LNetError: 39644:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg:0 not found [ 909.475196] LNet: Removed LNI 10.1.2.31@tcp [ 909.505780] LNet: Removed LNI 10.1.2.3@tcp [ 910.433446] Key type .llcrypt unregistered [ 910.437442] Key type ._llcrypt unregistered [ 930.246223] Key type ._llcrypt registered [ 930.250859] Key type .llcrypt registered [ 932.399193] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 08:33:53 (1789130033) [ 932.889535] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 932.908289] alg: No test for adler32 (adler32-zlib) [ 933.950425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 934.078288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 934.101318] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 934.795892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 934.812627] LNet: 40603:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 934.823866] LNetError: 40603:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 934.840923] LNet: Removed LNI 192.168.201.5@tcp [ 934.856526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 934.890802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --nid 192.168.201.5@tcp [ 934.905978] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 935.803509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --nid 192.168.201.5@tcp [ 935.873387] LNet: Removed LNI 192.168.201.5@tcp [ 935.977717] LNet: 40738:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 936.941693] Key type .llcrypt unregistered [ 936.947360] Key type ._llcrypt unregistered [ 952.062708] Key type ._llcrypt registered [ 952.064567] Key type .llcrypt registered [ 953.598426] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 08:34:15 (1789130055) [ 956.266851] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_111 Need at least 2 remote nodes found oleg105-server [ 958.929836] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 08:34:20 (1789130060) [ 959.846792] Key type .llcrypt unregistered [ 959.853696] Key type ._llcrypt unregistered [ 967.778834] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 968.058255] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 968.069151] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 968.071125] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 970.835553] Key type ._llcrypt registered [ 970.837641] Key type .llcrypt registered [ 971.109346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 971.127549] alg: No test for adler32 (adler32-zlib) [ 972.202862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 972.298513] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 973.188716] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180] [ 976.263532] LNet: 42583:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 976.267842] LNetError: 42583:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 976.271634] LNet: Removed LNI 192.168.201.5@tcp1 [ 976.281215] LNet: Removed LNI 10.1.2.3@tcp2 [ 977.214596] Key type .llcrypt unregistered [ 977.223841] Key type ._llcrypt unregistered [ 993.979420] Key type ._llcrypt registered [ 993.982319] Key type .llcrypt registered [ 995.581821] Lustre: DEBUG MARKER: == sanity-lnet test 150: Check import of ip2nets yaml sequence ========================================================== 08:34:57 (1789130097) [ 996.134619] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 996.150257] alg: No test for adler32 (adler32-zlib) [ 997.280670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 997.388840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 997.441088] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 998.610709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 998.652487] LNet: 43557:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 998.664641] LNetError: 43557:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 998.679248] LNet: Removed LNI 192.168.201.5@tcp [ 998.701373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 998.828411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet.yaml [ 998.905468] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1007.263860] Lustre: DEBUG MARKER: == sanity-lnet test 151: Check correct application of multiple ip2nets rules ========================================================== 08:35:08 (1789130108) [ 1007.395887] LNet: 44142:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1007.404947] LNetError: 44142:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1007.429603] LNet: Removed LNI 192.168.201.5@tcp [ 1008.126467] Key type .llcrypt unregistered [ 1008.131216] Key type ._llcrypt unregistered [ 1015.864523] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1016.186835] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1016.195407] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1016.207067] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1019.189624] Key type ._llcrypt registered [ 1019.194787] Key type .llcrypt registered [ 1019.646868] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1019.708476] alg: No test for adler32 (adler32-zlib) [ 1020.725192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1020.827184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1020.865576] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1021.748985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1021.773392] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1024.339951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1024.367432] LNet: 44894:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1024.378344] LNetError: 44894:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1024.386418] LNet: Removed LNI 10.1.2.3@tcp [ 1024.401247] LNet: Removed LNI 192.168.201.5@tcp [ 1024.432390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1024.490501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 1024.513408] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1024.520189] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1025.471852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1025.504258] LNet: 44921:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1025.516986] LNetError: 44921:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1025.525373] LNetError: 44921:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1025.533467] LNet: Removed LNI 10.1.2.3@tcp [ 1025.550970] LNet: Removed LNI 192.168.201.5@tcp [ 1025.578217] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1025.636531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 1025.662609] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1025.669610] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1026.315728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1026.332362] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 1027.637578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1027.678554] LNet: 45084:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1027.693896] LNetError: 45084:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1027.703870] LNetError: 45084:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1027.714487] LNet: Removed LNI 10.1.2.3@tcp [ 1027.720670] LNet: Removed LNI 192.168.201.5@tcp [ 1027.738560] LNet: Removed LNI 192.168.201.5@tcp2 [ 1027.784640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1027.878680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 1027.904428] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1027.908380] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1027.912917] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 1037.994882] Lustre: DEBUG MARKER: == sanity-lnet test 152: Check import of ip2nets with wildcard IP patterns ========================================================== 08:35:39 (1789130139) [ 1038.216389] LNet: 45681:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1038.220255] LNetError: 45681:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1038.224786] LNetError: 45681:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 1038.241135] LNet: Removed LNI 10.1.2.3@tcp [ 1038.244349] LNet: Removed LNI 192.168.201.5@tcp [ 1038.257554] LNet: Removed LNI 192.168.201.5@tcp2 [ 1039.088751] Key type .llcrypt unregistered [ 1039.090250] Key type ._llcrypt unregistered [ 1049.964824] Key type ._llcrypt registered [ 1049.970225] Key type .llcrypt registered [ 1050.375109] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1050.422833] alg: No test for adler32 (adler32-zlib) [ 1051.465882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1051.527975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1051.552597] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1052.514886] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1052.535972] LNet: 46080:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1052.544570] LNetError: 46080:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1052.563824] LNet: Removed LNI 192.168.201.5@tcp [ 1052.577694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1052.647853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1052.709427] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1053.596859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1053.615431] LNet: 46110:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1053.621424] LNetError: 46110:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1053.634379] LNet: Removed LNI 192.168.201.5@tcp [ 1053.650320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1053.701341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1053.737949] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1054.433967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1054.505582] LNet: Removed LNI 192.168.201.5@tcp [ 1054.551647] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1054.607534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1054.640462] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1055.209556] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1055.429233] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1055.434169] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1055.437544] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1057.819336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1057.839411] LNet: 46213:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1057.844500] LNet: 46213:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 1057.865127] LNetError: 46213:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1057.881338] LNetError: 46213:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1057.926696] LNet: Removed LNI 192.168.201.5@tcp [ 1057.950487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1058.069113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1058.114231] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1068.284984] Lustre: DEBUG MARKER: == sanity-lnet test 153: Check import of ip2nets with IP address ranges ========================================================== 08:36:09 (1789130169) [ 1068.455529] LNet: 46805:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1068.467281] LNetError: 46805:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1068.507307] LNet: Removed LNI 192.168.201.5@tcp [ 1069.599475] Key type .llcrypt unregistered [ 1069.605742] Key type ._llcrypt unregistered [ 1078.824166] Key type ._llcrypt registered [ 1078.826685] Key type .llcrypt registered [ 1079.012411] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1079.076691] alg: No test for adler32 (adler32-zlib) [ 1080.106344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1080.218269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1080.273776] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1081.711866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1081.742078] LNet: 47215:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1081.749921] LNetError: 47215:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1081.766674] LNet: Removed LNI 192.168.201.5@tcp [ 1081.787790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1081.861336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-153.yaml [ 1081.885596] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1090.205791] Lustre: DEBUG MARKER: == sanity-lnet test 154: Check import of ip2nets with stepped IP address ranges ========================================================== 08:36:31 (1789130191) [ 1090.279952] LNet: 47801:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1090.284512] LNetError: 47801:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1090.304825] LNet: Removed LNI 192.168.201.5@tcp [ 1091.259991] Key type .llcrypt unregistered [ 1091.263976] Key type ._llcrypt unregistered [ 1099.173479] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1099.364718] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1099.386584] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1099.404263] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1102.119326] Key type ._llcrypt registered [ 1102.126037] Key type .llcrypt registered [ 1102.324645] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1102.347276] alg: No test for adler32 (adler32-zlib) [ 1103.285059] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1103.345092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1103.367974] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1104.195568] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1104.211911] LNet: 48262:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1104.223270] LNetError: 48262:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1104.246502] LNet: Removed LNI 192.168.201.5@tcp [ 1104.269692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1104.372983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-154.yaml [ 1104.391820] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1111.770355] Lustre: DEBUG MARKER: == sanity-lnet test 155: Check ip2nets import failure with no matching rules ========================================================== 08:36:53 (1789130213) [ 1112.051469] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1112.076621] LNet: 48861:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1112.081716] LNetError: 48861:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1112.121480] LNet: Removed LNI 192.168.201.5@tcp [ 1112.141392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1112.285147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-155.yaml [ 1118.432648] Lustre: DEBUG MARKER: == sanity-lnet test 156: Check import of ip2nets with comma-separated IP ranges ========================================================== 08:37:00 (1789130220) [ 1118.621531] LNet: 49440:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1119.791485] Key type .llcrypt unregistered [ 1119.796763] Key type ._llcrypt unregistered [ 1126.988817] Key type ._llcrypt registered [ 1126.991707] Key type .llcrypt registered [ 1127.275858] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1127.292370] alg: No test for adler32 (adler32-zlib) [ 1128.228111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1128.295524] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1128.315613] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1129.549807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1129.585219] LNet: 49845:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1129.598532] LNetError: 49845:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1129.624197] LNet: Removed LNI 192.168.201.5@tcp [ 1129.650784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1129.794055] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-156.yaml [ 1129.817966] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1138.395902] Lustre: DEBUG MARKER: == sanity-lnet test 157: Check import failure with malformed ip2nets YAML ========================================================== 08:37:20 (1789130240) [ 1138.621340] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1138.661367] LNet: 50437:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1138.686674] LNetError: 50437:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1138.717792] LNet: Removed LNI 192.168.201.5@tcp [ 1138.730562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1138.810601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-157.yaml [ 1146.240885] Lustre: DEBUG MARKER: == sanity-lnet test 158: Check import of ip2nets with single IPv6 address ========================================================== 08:37:27 (1789130247) [ 1147.577379] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_158 Interface has no IPv6 [ 1149.502653] Lustre: DEBUG MARKER: == sanity-lnet test 159: Check import of ip2nets with IPv6 /128 CIDR notation ========================================================== 08:37:31 (1789130251) [ 1151.540228] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_159 Interface has no IPv6 [ 1154.068245] Lustre: DEBUG MARKER: == sanity-lnet test 160: Check correct application of multiple IPv6 ip2nets rules ========================================================== 08:37:35 (1789130255) [ 1156.087581] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_160 Interface has no IPv6 [ 1157.822184] Lustre: DEBUG MARKER: == sanity-lnet test 161: Check import of ip2nets with various IPv6 prefix lengths ========================================================== 08:37:39 (1789130259) [ 1159.906562] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_161 Interface has no IPv6 [ 1161.947249] Lustre: DEBUG MARKER: == sanity-lnet test 162: Check ip2nets import failure with non-matching IPv6 ========================================================== 08:37:43 (1789130263) [ 1163.671206] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_162 Interface has no IPv6 [ 1165.462639] Lustre: DEBUG MARKER: == sanity-lnet test 170: Check CPTs and tunables parsing when importing ========================================================== 08:37:47 (1789130267) [ 1165.805324] LNet: 52490:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1166.800877] Key type .llcrypt unregistered [ 1166.807804] Key type ._llcrypt unregistered [ 1175.141031] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1175.332722] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1175.340995] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1175.361264] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1176.976315] Key type ._llcrypt registered [ 1176.980885] Key type .llcrypt registered [ 1177.201588] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1177.215646] alg: No test for adler32 (adler32-zlib) [ 1178.151857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1178.220644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1178.239930] LNet: Added LNI 192.168.201.5@tcp [8/512/0/180] [ 1178.248133] LNet: Added LNI 10.1.2.3@tcp [8/512/0/180] [ 1178.252731] LNet: 52806:0:(api-ni.c:2986:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1178.374717] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1178.391498] LNet: 52825:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1178.396828] LNetError: 52825:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1178.402824] LNet: Removed LNI 10.1.2.3@tcp [ 1178.405488] LNetError: 52825:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1178.424684] LNet: Removed LNI 192.168.201.5@tcp [ 1178.460615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1178.547778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1178.572881] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1178.649499] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1178.674258] LNet: 52849:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1178.685456] LNetError: 52849:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1178.699740] LNet: Removed LNI 192.168.201.5@tcp [ 1178.731639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1178.806152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1178.831234] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1178.835815] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1178.839039] LNet: 52857:0:(api-ni.c:2986:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1178.974363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1179.015926] LNet: 52875:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1179.021981] LNetError: 52875:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1179.030667] LNet: Removed LNI 10.1.2.3@tcp [ 1179.036366] LNetError: 52875:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1179.054747] LNet: Removed LNI 192.168.201.5@tcp [ 1179.082378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1179.210717] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1179.242422] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1179.251949] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1179.265055] LNet: 52883:0:(api-ni.c:2986:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1179.481786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1179.534036] LNet: 52909:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1179.540754] LNetError: 52909:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1179.548599] LNet: Removed LNI 10.1.2.3@tcp [ 1179.553900] LNetError: 52909:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1179.570925] LNet: Removed LNI 192.168.201.5@tcp [ 1179.608271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1179.667712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1179.707672] LNet: Added LNI 192.168.201.5@tcp [8/256/0/182] [ 1179.716172] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 1179.865728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1179.892544] LNet: 52938:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1179.912856] LNetError: 52938:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1179.932203] LNet: Removed LNI 10.1.2.3@tcp [ 1179.936928] LNetError: 52938:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1179.964926] LNet: Removed LNI 192.168.201.5@tcp [ 1180.016347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1180.128831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1180.149954] LNet: Added LNI 192.168.201.5@tcp [8/256/0/182] [ 1180.159331] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 1180.161420] LNet: 52946:0:(api-ni.c:2986:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1180.242515] LNet: Removed LNI 10.1.2.3@tcp [ 1180.251481] LNet: Removed LNI 192.168.201.5@tcp [ 1181.065890] Key type .llcrypt unregistered [ 1181.067506] Key type ._llcrypt unregistered [ 1196.313575] Key type ._llcrypt registered [ 1196.317229] Key type .llcrypt registered [ 1197.780792] Lustre: DEBUG MARKER: == sanity-lnet test 180: FastReg map error should not leak descriptors ========================================================== 08:38:19 (1789130299) [ 1199.381520] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_180 Need o2ib NETTYPE [ 1201.236130] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 08:38:23 (1789130303) [ 1201.751565] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1201.768211] alg: No test for adler32 (adler32-zlib) [ 1202.790950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1202.864734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1202.907342] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1203.903510] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --ip2net tcp 192.168.201.5 [ 1203.918496] LNet: 54214:0:(config.c:1738:lnet_parse_ip2nets()) Kernel parsing of ip2nets is deprecated. Consider converting to LNet YAML configuration. [ 1203.924940] LNet: Removed LNI 192.168.201.5@tcp [ 1211.438508] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 08:38:33 (1789130313) [ 1211.768193] LNet: 54784:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1211.777266] LNetError: 54784:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1211.803225] LNet: Removed LNI 192.168.201.5@tcp [ 1212.693382] Key type .llcrypt unregistered [ 1212.694768] Key type ._llcrypt unregistered [ 1221.258625] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1221.474410] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1224.095911] Key type ._llcrypt registered [ 1224.098455] Key type .llcrypt registered [ 1224.346086] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1224.367512] alg: No test for adler32 (adler32-zlib) [ 1225.683909] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1234.334240] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 08:38:56 (1789130336) [ 1234.508478] LNet: 55693:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1234.516870] LNetError: 55693:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1234.535007] LNet: Removed LNI 10.1.2.3@tcp [ 1235.316457] Key type .llcrypt unregistered [ 1235.322231] Key type ._llcrypt unregistered [ 1243.507697] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1243.798899] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1243.804624] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1243.808112] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1246.559846] Key type ._llcrypt registered [ 1246.562168] Key type .llcrypt registered [ 1246.750345] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1246.791372] alg: No test for adler32 (adler32-zlib) [ 1247.742534] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1253.375307] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 08:39:15 (1789130355) [ 1255.007483] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_202 kernel ip2nets deprecated in 2.17.53 [ 1256.883630] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 08:39:18 (1789130358) [ 1257.228427] LNet: 56890:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1257.235199] LNetError: 56890:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1257.252956] LNet: Removed LNI 10.1.2.3@tcp [ 1257.982424] Key type .llcrypt unregistered [ 1257.984112] Key type ._llcrypt unregistered [ 1264.651826] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1264.856743] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1264.860146] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1264.862586] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1267.721413] Key type ._llcrypt registered [ 1267.728404] Key type .llcrypt registered [ 1267.993570] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1268.014980] alg: No test for adler32 (adler32-zlib) [ 1269.015795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1269.081240] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1278.765851] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 08:39:40 (1789130380) [ 1279.375717] LNet: 58123:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1279.385980] LNetError: 58123:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1279.402297] LNet: Removed LNI 10.1.2.3@tcp [ 1280.193940] Key type .llcrypt unregistered [ 1280.196609] Key type ._llcrypt unregistered [ 1288.658906] Key type ._llcrypt registered [ 1288.661648] Key type .llcrypt registered [ 1289.022889] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1289.047353] alg: No test for adler32 (adler32-zlib) [ 1290.034365] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1290.092201] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1298.214508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1299.204725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_interrupt -r 1 [ 1299.258450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_interrupt -r 1 [ 1299.328846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1299.388177] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1299.730289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1299.785474] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1299.817503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1299.866073] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1299.901791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1299.953572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1300.008777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1300.067834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1300.110446] LNet: 58565:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1300.149695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1300.184443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1300.243122] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1300.285828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1300.314901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1300.381679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1300.395650] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1300.410314] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1300.456271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1301.160574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_dropped -r 1 [ 1301.200026] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_dropped -r 1 [ 1301.244845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1301.694668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1301.721761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1301.750963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1301.787578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1301.819104] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1301.855709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1301.907261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1301.951924] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1302.001183] LNet: 58674:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1302.061679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1302.119456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1302.161284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1302.209964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1302.247567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1302.277425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1302.331801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1303.149147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_aborted -r 1 [ 1303.208063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_aborted -r 1 [ 1303.293356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1303.324908] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1303.333022] LNet: Skipped 1 previous similar message [ 1303.652039] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1303.685511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1303.729297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1303.796665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1303.866346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1303.922763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1303.976189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1304.024433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1304.046290] LNet: 58784:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1304.076973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1304.124946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1304.177242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1304.225575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1304.262689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1304.306721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1304.361043] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1305.017843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_no_route -r 1 [ 1305.066488] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_no_route -r 1 [ 1305.096715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1305.343450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1305.375224] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1305.417157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1305.459109] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1305.503270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1305.541913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1305.578271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1305.610884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1305.658357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1305.697949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1305.728375] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1305.758582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1305.783659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1305.807894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1305.833685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1306.344479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_timeout -r 1 [ 1306.401475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_timeout -r 1 [ 1306.439782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1306.472958] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1306.487322] LNet: Skipped 1 previous similar message [ 1306.739858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1306.789495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1306.832668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1306.881365] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1306.928334] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1306.963266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1307.003265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1307.041496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1307.074650] LNet: 59002:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1307.079971] LNet: 59002:0:(api-ni.c:296:recovery_interval_set()) Skipped 1 previous similar message [ 1307.097300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1307.139837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1307.171990] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1307.212408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1307.241500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1307.279181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1307.307286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1307.805486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_error -r 1 [ 1307.838410] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_error -r 1 [ 1307.865302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1308.065401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1308.096432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1308.123471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1308.151816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1308.177799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1308.201820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1308.223426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1308.250764] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1308.283194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1308.309641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1308.344834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1308.377755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1308.411279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1308.442909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1308.479464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1308.816062] LNet: 59156:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1308.831324] LNetError: 59156:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1309.871246] LNet: Removed LNI 192.168.201.5@tcp [ 1310.401544] Key type .llcrypt unregistered [ 1310.404281] Key type ._llcrypt unregistered [ 1331.390759] Key type ._llcrypt registered [ 1331.392795] Key type .llcrypt registered [ 1332.874591] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 08:40:34 (1789130434) [ 1333.892559] Key type .llcrypt unregistered [ 1333.894202] Key type ._llcrypt unregistered [ 1340.447492] Key type ._llcrypt registered [ 1340.450295] Key type .llcrypt registered [ 1340.744459] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1340.767440] alg: No test for adler32 (adler32-zlib) [ 1341.750301] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1341.787519] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1350.924696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1363.510838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1363.559127] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1363.584150] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1365.226643] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_interrupt -r 1 [ 1365.281299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_interrupt -r 1 [ 1365.332452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_interrupt -r 1 [ 1365.371710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_interrupt -r 1 [ 1365.405558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_interrupt -r 1 [ 1365.434433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_interrupt -r 1 [ 1365.468569] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_interrupt -r 1 [ 1365.499933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_interrupt -r 1 [ 1365.547467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1365.569659] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1365.576462] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1365.813032] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1365.857567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1365.919961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1365.966508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1366.013569] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1366.070510] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1366.123522] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1366.187526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1366.215246] LNet: 60629:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1366.239340] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1366.283896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1366.334819] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1366.396171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1366.452906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1366.511280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1366.571785] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1367.238430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_dropped -r 1 [ 1367.281359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_dropped -r 1 [ 1367.354400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_dropped -r 1 [ 1367.411269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_dropped -r 1 [ 1367.472428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_dropped -r 1 [ 1367.509352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_dropped -r 1 [ 1367.547737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_dropped -r 1 [ 1367.576514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_dropped -r 1 [ 1367.603262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1367.620798] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1367.625172] LNet: Skipped 2 previous similar messages [ 1367.901446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1367.945869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1367.988764] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1368.033892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1368.078297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1368.118293] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1368.148263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1368.181265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1368.212269] LNet: 60750:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1368.231875] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1368.276703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1368.322995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1368.375951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1368.425605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1368.480568] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1368.535604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1369.176350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_aborted -r 1 [ 1369.218433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_aborted -r 1 [ 1369.256365] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_aborted -r 1 [ 1369.293573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_aborted -r 1 [ 1369.328728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_aborted -r 1 [ 1369.366354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_aborted -r 1 [ 1369.397651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_aborted -r 1 [ 1369.429517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_aborted -r 1 [ 1369.463873] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1369.486306] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1369.490919] LNet: Skipped 2 previous similar messages [ 1369.714620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1369.739542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1369.772892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1369.805212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1369.832810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1369.862590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1369.895675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1369.931953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1369.948465] LNet: 60871:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1369.970897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1370.003745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1370.036567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1370.067392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1370.097529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1370.128867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1370.185967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1371.012596] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_no_route -r 1 [ 1371.074305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_no_route -r 1 [ 1371.115870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_no_route -r 1 [ 1371.187800] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_no_route -r 1 [ 1371.278408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_no_route -r 1 [ 1371.361264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_no_route -r 1 [ 1371.424734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_no_route -r 1 [ 1371.471173] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_no_route -r 1 [ 1371.520582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1371.553505] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1371.559771] LNet: Skipped 2 previous similar messages [ 1371.908284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1371.951835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1372.001471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1372.038779] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1372.081659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1372.126421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1372.220653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1372.292781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1372.312881] LNet: 60992:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1372.357575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1372.418237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1372.487748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1372.568819] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1372.643223] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1372.708080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1372.782572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1373.518595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_timeout -r 1 [ 1373.593963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_timeout -r 1 [ 1373.669644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_timeout -r 1 [ 1373.711397] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_timeout -r 1 [ 1373.788960] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_timeout -r 1 [ 1373.863424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_timeout -r 1 [ 1373.948888] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_timeout -r 1 [ 1374.047250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_timeout -r 1 [ 1374.154419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1374.602869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1374.670181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1374.758875] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1374.819120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1374.896150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1374.962222] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1375.015252] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1375.072155] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1375.126617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1375.185158] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1375.258539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1375.302995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1375.325387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1375.380859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1375.436492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1376.241862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_error -r 1 [ 1376.292081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_error -r 1 [ 1376.324776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_error -r 1 [ 1376.359857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_error -r 1 [ 1376.408676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_error -r 1 [ 1376.449597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_error -r 1 [ 1376.490357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_error -r 1 [ 1376.522761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_error -r 1 [ 1376.550651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1376.570596] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1376.575581] LNet: Skipped 7 previous similar messages [ 1376.772270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1376.797965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1376.822827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1376.868869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1376.919282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1376.957666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1377.001871] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1377.042767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1377.059118] LNet: 61235:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1377.069611] LNet: 61235:0:(api-ni.c:296:recovery_interval_set()) Skipped 1 previous similar message [ 1377.111052] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1377.152260] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1377.201368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1377.231607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1377.265603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1377.290876] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1377.323758] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1384.868493] LNet: 61315:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1384.875681] LNetError: 61315:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1385.888415] LNet: Removed LNI 192.168.201.5@tcp [ 1385.894735] LNetError: 61315:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1385.923660] LNet: Removed LNI 192.168.201.5@tcp1 [ 1386.661672] Key type .llcrypt unregistered [ 1386.667236] Key type ._llcrypt unregistered [ 1410.578473] Key type ._llcrypt registered [ 1410.580030] Key type .llcrypt registered [ 1411.916136] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 08:41:53 (1789130513) [ 1413.257648] Key type .llcrypt unregistered [ 1413.259658] Key type ._llcrypt unregistered [ 1421.049600] Key type ._llcrypt registered [ 1421.051843] Key type .llcrypt registered [ 1421.214027] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1421.233646] alg: No test for adler32 (adler32-zlib) [ 1422.146262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1422.172221] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1430.162587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1430.700419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_dropped -r 1 [ 1430.723163] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_dropped -r 1 [ 1430.757245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1430.779153] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1431.089327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1431.136954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1431.165726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1431.194581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1431.213270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1431.249417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1431.276863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1431.320966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1431.339788] LNet: 62570:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1431.358853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1431.420292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1431.452345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1431.469108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1431.486149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1431.523091] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1431.573807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1432.410953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_error -r 1 [ 1432.455242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_error -r 1 [ 1432.514248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1432.540298] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1432.749689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1432.778344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1432.805310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1432.830203] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1432.857768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1432.885854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1432.911989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1432.939823] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1432.955869] LNet: 62679:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1432.970188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1432.996358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1433.027333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1433.052914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1433.090173] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1433.133154] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1433.179221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1433.861488] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_timeout -r 1 [ 1433.909810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_timeout -r 1 [ 1433.939922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1433.953586] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1434.285412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1434.320303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1434.346472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1434.369363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1434.391658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1434.416585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1434.439343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1434.474853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1434.493519] LNet: 62789:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1434.508815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1434.535720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1434.571195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1434.605541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1434.645762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1434.668738] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1434.699400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1434.950800] LNet: 62834:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1434.954583] LNetError: 62834:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1434.967723] LNet: Removed LNI 192.168.201.5@tcp [ 1435.365946] Key type .llcrypt unregistered [ 1435.376616] Key type ._llcrypt unregistered [ 1456.688562] Key type ._llcrypt registered [ 1456.693222] Key type .llcrypt registered [ 1457.918533] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 08:42:39 (1789130559) [ 1459.628729] Key type .llcrypt unregistered [ 1459.632568] Key type ._llcrypt unregistered [ 1466.509946] Key type ._llcrypt registered [ 1466.513869] Key type .llcrypt registered [ 1466.738513] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1466.744682] alg: No test for adler32 (adler32-zlib) [ 1467.681903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1467.713108] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1477.102843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1488.087865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1488.162210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1488.208574] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1489.751986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_dropped -r 1 [ 1489.815039] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e remote_dropped -r 1 [ 1489.848632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_dropped -r 1 [ 1489.883972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e remote_dropped -r 1 [ 1489.929877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e remote_dropped -r 1 [ 1489.968319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e remote_dropped -r 1 [ 1490.005923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e remote_dropped -r 1 [ 1490.043973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e remote_dropped -r 1 [ 1490.083878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1490.107307] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1490.112377] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.105@tcp [ 1490.301359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1490.334258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1490.372417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1490.416621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1490.468924] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1490.506263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1490.538200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1490.564654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1490.582215] LNet: 64308:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1490.604705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1490.630353] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1490.660968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1490.693633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1490.722605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1490.757572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1490.787752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1490.826903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1491.372303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_error -r 1 [ 1491.404611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e remote_error -r 1 [ 1491.463800] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_error -r 1 [ 1491.537329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e remote_error -r 1 [ 1491.575545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e remote_error -r 1 [ 1491.656365] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e remote_error -r 1 [ 1491.689743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e remote_error -r 1 [ 1491.719718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e remote_error -r 1 [ 1491.755228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1491.776417] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1491.780597] LNet: Skipped 2 previous similar messages [ 1491.964584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1491.997592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1492.026256] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1492.057748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1492.087303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1492.132787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1492.181554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1492.243832] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1492.262390] LNet: 64431:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1492.289608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1492.325734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1492.359605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1492.398388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1492.439737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1492.481826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1492.543072] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1492.572895] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1493.063866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_timeout -r 1 [ 1493.100385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e remote_timeout -r 1 [ 1493.135953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_timeout -r 1 [ 1493.177177] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e remote_timeout -r 1 [ 1493.214357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e remote_timeout -r 1 [ 1493.257612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e remote_timeout -r 1 [ 1493.301391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e remote_timeout -r 1 [ 1493.343861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e remote_timeout -r 1 [ 1493.420551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1493.442679] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1493.446839] LNet: Skipped 1 previous similar message [ 1493.649742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1493.677750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1493.703358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1493.728105] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1493.759668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1493.788337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1493.814248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1493.852836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1493.867874] LNet: 64554:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1493.883758] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1493.922916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1493.955513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1493.987880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1494.017534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1494.041955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1494.068779] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1494.094863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1499.419583] LNet: 64636:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1499.431532] LNetError: 64636:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1499.444430] LNet: Removed LNI 192.168.201.5@tcp [ 1499.472386] LNet: Removed LNI 192.168.201.5@tcp1 [ 1500.010977] Key type .llcrypt unregistered [ 1500.018174] Key type ._llcrypt unregistered [ 1519.614832] Key type ._llcrypt registered [ 1519.617223] Key type .llcrypt registered [ 1520.922874] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 08:43:42 (1789130622) [ 1523.008052] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_208 kernel ip2nets deprecated in 2.17.53 [ 1524.930786] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 08:43:46 (1789130626) [ 1526.276410] Key type .llcrypt unregistered [ 1526.279423] Key type ._llcrypt unregistered [ 1533.336704] Key type ._llcrypt registered [ 1533.338418] Key type .llcrypt registered [ 1533.699284] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1533.743881] alg: No test for adler32 (adler32-zlib) [ 1534.686575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1534.742880] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1543.307821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1544.190455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e network_timeout -r 1 [ 1544.249987] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e network_timeout -r 1 [ 1544.294098] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1544.329354] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1544.643719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1544.676598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1544.710249] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1544.748510] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1544.782198] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1544.849937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1544.901273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1544.985235] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1544.992717] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1545.034041] LNet: 66185:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1545.081143] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1545.151693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1545.221659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1545.293890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1545.386790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1545.446675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1545.957157] LNet: 66228:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1545.966295] LNetError: 66228:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1545.982760] LNet: Removed LNI 192.168.201.5@tcp [ 1546.744658] Key type .llcrypt unregistered [ 1546.754668] Key type ._llcrypt unregistered [ 1568.679323] Key type ._llcrypt registered [ 1568.682725] Key type .llcrypt registered [ 1569.044563] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1569.059501] alg: No test for adler32 (adler32-zlib) [ 1570.099861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1570.137612] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1579.309384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1590.401968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1590.512141] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1590.553980] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1593.048996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e network_timeout -r 1 [ 1593.125819] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e network_timeout -r 1 [ 1593.218605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e network_timeout -r 1 [ 1593.297208] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e network_timeout -r 1 [ 1593.361830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e network_timeout -r 1 [ 1593.428380] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e network_timeout -r 1 [ 1593.484866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e network_timeout -r 1 [ 1593.524867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e network_timeout -r 1 [ 1593.589425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1593.666589] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1593.696804] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1593.706287] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.105@tcp [ 1594.205631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1594.274817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1594.352847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1594.421342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1594.466245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1594.516475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1594.575799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1594.658520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1594.712716] LNet: 67112:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1594.721807] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1594.749307] LNet: Skipped 1 previous similar message [ 1594.788390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1594.847583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1594.879297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1594.917333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1594.957111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1595.021887] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1600.519815] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1600.988906] LNet: 67190:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1600.997893] LNetError: 67190:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1602.016420] LNet: Removed LNI 192.168.201.5@tcp [ 1602.018418] LNetError: 67190:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1602.056861] LNet: Removed LNI 192.168.201.5@tcp1 [ 1602.702897] Key type .llcrypt unregistered [ 1602.705114] Key type ._llcrypt unregistered [ 1625.017860] Key type ._llcrypt registered [ 1625.020660] Key type .llcrypt registered [ 1627.151417] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 08:45:28 (1789130728) [ 1627.734618] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1627.756548] alg: No test for adler32 (adler32-zlib) [ 1628.823509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1628.950562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1628.984979] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1630.291213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1630.317499] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1631.492603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 1631.617243] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 1631.693909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 1631.770443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 1631.849595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 1631.934757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1632.009186] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.5@tcp [ 1632.050436] LNetError: 68291:0:(api-ni.c:4473:lnet_ni_set_healthv()) manually adding local NI 192.168.201.5@tcp to recovery [ 1632.994902] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1632.996770] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1634.016431] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1634.659710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1638.112319] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1638.355368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1638.421501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1638.538631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1638.591777] LNet: 68388:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1638.598849] LNetError: 68388:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1638.611569] LNet: Removed LNI 192.168.201.5@tcp [ 1638.631924] LNet: Removed LNI 192.168.201.5@tcp1 [ 1638.675858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1638.771932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1638.830073] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1639.862865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1639.902543] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1641.097733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 1641.158232] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1641.268341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 1641.318545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 1641.384797] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 1641.449679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 1641.496391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1641.550261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.5@tcp [ 1641.575954] LNetError: 68660:0:(api-ni.c:4473:lnet_ni_set_healthv()) manually adding local NI 192.168.201.5@tcp to recovery [ 1641.825498] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1644.179863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1647.969141] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1647.981697] LNet: Skipped 1 previous similar message [ 1651.625175] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1651.681440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1651.756755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1660.978854] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 08:46:02 (1789130762) [ 1661.103729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1661.141399] LNet: 69350:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1661.150973] LNetError: 69350:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1661.157898] LNetError: 69350:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1661.163084] LNet: Removed LNI 192.168.201.5@tcp [ 1661.174266] LNet: Removed LNI 192.168.201.5@tcp1 [ 1661.203377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1661.279837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1661.318602] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1662.426849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1662.474659] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1663.420324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 1663.506861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 1663.547649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 1663.581086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 1663.617848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 1663.651725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1663.683864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.5@tcp --health 0 [ 1663.817983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1664.289134] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.5@tcp [ 1664.292390] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1664.292400] LNet: Skipped 1 previous similar message [ 1678.893259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1679.068907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1679.114988] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1679.142801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.5@tcp --health 500 [ 1679.200080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1686.125233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1686.255856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1686.312125] LNet: 69841:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1686.323780] LNetError: 69841:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1686.357408] LNetError: 69841:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1686.365757] LNet: Removed LNI 192.168.201.5@tcp [ 1686.391220] LNet: Removed LNI 192.168.201.5@tcp1 [ 1686.440122] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1686.609063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1686.673574] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1688.091847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1688.136553] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1689.042965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 1689.086420] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1689.185856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 1689.242970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 1689.306496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 1689.383331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1689.497373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1689.604060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.5@tcp --health 0 [ 1690.657132] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1690.667827] LNet: Skipped 3 previous similar messages [ 1691.110456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1702.264048] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1702.351878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1702.436540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1710.662619] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 08:46:51 (1789130811) [ 1711.543593] LNet: 70889:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1711.551528] LNetError: 70889:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1711.567431] LNetError: 70889:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1711.578807] LNet: Removed LNI 192.168.201.5@tcp [ 1711.601571] LNet: Removed LNI 192.168.201.5@tcp1 [ 1712.533532] Key type .llcrypt unregistered [ 1712.540082] Key type ._llcrypt unregistered [ 1721.052588] Key type ._llcrypt registered [ 1721.055940] Key type .llcrypt registered [ 1721.434963] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1721.468832] alg: No test for adler32 (adler32-zlib) [ 1722.454355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1722.498517] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1734.419474] Lustre: DEBUG MARKER: Initial discovery [ 1734.757383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.105@tcp [ 1737.732612] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag [ 1737.841771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 -e local_error [ 1737.901725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.105@tcp [ 1737.968188] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1738.006911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1738.089306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.201.105@tcp -r 1 -m GET -l 3 [ 1738.181369] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.201.105@tcp -r 1 -m GET -e local_error [ 1738.239927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.201.105@tcp -r 1 -m PUT -l 6 [ 1738.912557] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1739.894863] Lustre: DEBUG MARKER: Force oleg105-client.virtnet to discover 192.168.201.105@tcp (in background) [ 1740.311067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.105@tcp [ 1742.075416] Lustre: DEBUG MARKER: Force oleg105-server to discover 192.168.201.5@tcp [ 1743.392493] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1744.758500] Lustre: DEBUG MARKER: Wait for 71658 [ 1746.282599] Lustre: DEBUG MARKER: Finished wait on 71658 [ 1746.672736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay del -a [ 1746.733765] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1746.761058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1747.121414] LNet: 72110:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1747.130823] LNetError: 72110:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1748.200925] LNet: Removed LNI 192.168.201.5@tcp [ 1748.663511] Key type .llcrypt unregistered [ 1748.668961] Key type ._llcrypt unregistered [ 1773.092695] Key type ._llcrypt registered [ 1773.094991] Key type .llcrypt registered [ 1775.119289] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 08:47:56 (1789130876) [ 1776.049710] Key type .llcrypt unregistered [ 1776.050901] Key type ._llcrypt unregistered [ 1784.730299] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1785.056122] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1785.081216] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1785.122479] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1787.796883] Key type ._llcrypt registered [ 1787.802255] Key type .llcrypt registered [ 1788.073882] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1788.109688] alg: No test for adler32 (adler32-zlib) [ 1789.108789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1789.208181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1789.265327] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1790.274923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1790.313584] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1800.834972] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 08:48:22 (1789130902) [ 1800.994190] LNet: 74191:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1801.008368] LNetError: 74191:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1801.021264] LNet: Removed LNI 10.1.2.3@tcp [ 1801.049740] LNet: Removed LNI 192.168.201.5@tcp [ 1801.876901] Key type .llcrypt unregistered [ 1801.885478] Key type ._llcrypt unregistered [ 1810.557135] Key type ._llcrypt registered [ 1810.561964] Key type .llcrypt registered [ 1810.964370] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1810.982520] alg: No test for adler32 (adler32-zlib) [ 1812.000750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1812.076370] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1812.107886] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1812.757682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1812.787412] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1814.003993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 10.1.2.3@tcp 192.168.201.5@tcp [ 1815.072348] LNetError: Unexpected error -22 connecting to 192.168.201.5@tcp at host 192.168.201.5:988 [ 1823.631679] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 08:48:44 (1789130924) [ 1823.775618] LNet: 75329:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1823.808709] LNetError: 75329:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1823.825474] LNet: Removed LNI 10.1.2.3@tcp [ 1823.883367] LNet: Removed LNI 192.168.201.5@tcp [ 1824.736613] Key type .llcrypt unregistered [ 1824.739246] Key type ._llcrypt unregistered [ 1834.648048] Key type ._llcrypt registered [ 1834.652601] Key type .llcrypt registered [ 1835.119056] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1835.142933] alg: No test for adler32 (adler32-zlib) [ 1836.233465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1836.359500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1836.444776] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1837.615666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1837.640446] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 1838.570868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 192.168.201.5@tcp1 --nid 192.168.201.5@tcp2 [ 1843.282265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp1 192.168.201.5@tcp2 [ 1843.314425] LNetError: 76177:0:(lib-move.c:2101:lnet_handle_find_routed_path()) Unable to send message from 192.168.201.5@tcp1 to 192.168.201.5@tcp2 - Route table may be misconfigured [ 1843.344792] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp2 192.168.201.5@tcp1 [ 1850.702876] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 08:49:12 (1789130952) [ 1851.013332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1851.062262] LNet: 76753:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1851.066794] LNetError: 76753:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1851.070262] LNet: Removed LNI 192.168.201.5@tcp1 [ 1852.135229] LNet: Removed LNI 192.168.201.5@tcp2 [ 1852.177064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1852.242721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1852.291676] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1853.037495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1853.059652] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1853.882541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 1853.930145] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.5@tcp [ 1853.958745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp1 192.168.201.5@tcp1 [ 1854.002606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.5@tcp -d 192.168.201.5@tcp -e network_timeout [ 1854.042877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -e network_timeout [ 1854.109633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -e network_timeout [ 1854.153677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -e network_timeout [ 1854.199901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1854.221654] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1854.242109] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1854.252609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1854.355799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1854.475598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1862.498646] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 08:49:24 (1789130964) [ 1862.736766] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1862.783163] LNet: 77615:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1862.802386] LNetError: 77615:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1862.813666] LNetError: 77615:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1862.825912] LNet: Removed LNI 192.168.201.5@tcp [ 1863.865539] LNet: Removed LNI 192.168.201.5@tcp1 [ 1863.893001] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1864.023595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 0@lo [ 1864.445153] LNet: 77656:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1865.078520] Key type .llcrypt unregistered [ 1865.080791] Key type ._llcrypt unregistered [ 1880.032245] Key type ._llcrypt registered [ 1880.033929] Key type .llcrypt registered [ 1882.038286] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 08:49:43 (1789130983) [ 1882.515962] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1882.528880] alg: No test for adler32 (adler32-zlib) [ 1883.555192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1885.293581] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces [ 1887.627790] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 08:49:48 (1789130988) [ 1887.744976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1887.785352] LNet: 78747:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1887.821816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1887.925147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1888.008735] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1889.234966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1889.262291] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1890.817718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1890.880093] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp1 [ 1890.966467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp1 [ 1899.292757] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 08:50:01 (1789131001) [ 1901.294529] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_220 Need at least 2 remote nodes found oleg105-server [ 1903.906977] Lustre: DEBUG MARKER: == sanity-lnet test 221: Add routes w/DD disabled - check aliveness ========================================================== 08:50:05 (1789131005) [ 1905.987894] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_221 Need at least 2 remote nodes found oleg105-server [ 1908.397875] Lustre: DEBUG MARKER: == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 08:50:09 (1789131009) [ 1910.717045] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_222 Need at least 2 remote nodes found oleg105-server [ 1913.100361] Lustre: DEBUG MARKER: == sanity-lnet test 223: Check avoid_asym_router_failure=1 w/DD disabled ========================================================== 08:50:14 (1789131014) [ 1915.918079] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_223 Need at least 2 remote nodes found oleg105-server [ 1918.063642] Lustre: DEBUG MARKER: == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 08:50:19 (1789131019) [ 1921.101035] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_224 Need at least 2 remote nodes found oleg105-server [ 1923.477519] Lustre: DEBUG MARKER: == sanity-lnet test 225: Check avoid_asym_router_failure=0 w/DD disabled ========================================================== 08:50:24 (1789131024) [ 1925.881763] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_225 Need at least 2 remote nodes found oleg105-server [ 1927.969570] Lustre: DEBUG MARKER: == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 08:50:29 (1789131029) [ 1930.331224] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_226 Need at least 3 remote nodes found oleg105-server [ 1932.395805] Lustre: DEBUG MARKER: == sanity-lnet test 227: Check router peer health w/DD disabled ========================================================== 08:50:34 (1789131034) [ 1934.741790] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_227 Need at least 3 remote nodes found oleg105-server [ 1937.245798] Lustre: DEBUG MARKER: == sanity-lnet test 228: Routes should stay up when health is decremented ========================================================== 08:50:38 (1789131038) [ 1939.466627] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_228 Need at least 3 remote nodes found oleg105-server [ 1941.499320] Lustre: DEBUG MARKER: == sanity-lnet test 230: Test setting conns-per-peer ===== 08:50:43 (1789131043) [ 1941.807951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1941.846945] LNet: 82633:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1941.865519] LNetError: 82633:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1941.874799] LNet: Removed LNI 192.168.201.5@tcp [ 1942.957636] LNet: Removed LNI 192.168.201.5@tcp1 [ 1943.000966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1943.096735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1943.134662] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1944.029474] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 4 [ 1944.183517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1944.432188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1944.463481] LNet: 82781:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1944.468662] LNetError: 82781:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1944.473990] LNetError: 82781:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1944.493193] LNet: Removed LNI 192.168.201.5@tcp [ 1944.540926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1944.608657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 5 [ 1944.644087] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1945.503770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1945.659934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1945.676227] LNetError: 82926:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1945.698958] LNet: Removed LNI 192.168.201.5@tcp [ 1945.737433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1945.786856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1945.815651] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1946.565137] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 6 [ 1946.671687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1946.955952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1947.019864] LNet: 83073:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1947.034349] LNet: 83073:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 1948.070815] LNet: Removed LNI 192.168.201.5@tcp [ 1948.103829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1948.199386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 7 [ 1948.266886] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1949.613493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1950.004594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1950.056829] LNetError: 83218:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1950.062192] LNetError: 83218:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1950.090191] LNet: Removed LNI 192.168.201.5@tcp [ 1950.140285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1950.274776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1950.308852] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1951.411817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 8 [ 1951.540406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1951.848666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1951.882542] LNet: 83366:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1951.891031] LNet: 83366:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 1952.939725] LNet: Removed LNI 192.168.201.5@tcp [ 1952.978118] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1953.118938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 9 [ 1953.169255] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1954.400396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1954.666496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1954.713665] LNetError: 83511:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1954.720236] LNetError: 83511:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1955.761579] LNet: Removed LNI 192.168.201.5@tcp [ 1955.818307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1955.964465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1956.007911] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1956.702493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 10 [ 1956.764252] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1956.942564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1957.990292] LNet: Removed LNI 192.168.201.5@tcp [ 1958.072911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1958.153863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 11 [ 1958.181830] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1959.148419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1959.332377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1960.425610] LNet: Removed LNI 192.168.201.5@tcp [ 1960.446660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1960.548492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1960.610129] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1961.510409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 12 [ 1961.596267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1961.786885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1961.813740] LNet: 83951:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1961.818106] LNet: 83951:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 1962.860312] LNet: Removed LNI 192.168.201.5@tcp [ 1962.935395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1962.994324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 13 [ 1963.027567] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1963.816868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1964.144279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1964.181810] LNetError: 84097:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1964.189896] LNetError: 84097:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 1965.230314] LNet: Removed LNI 192.168.201.5@tcp [ 1965.310408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1965.406882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1965.438613] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1966.400200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 14 [ 1966.478770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1966.864852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1967.987869] LNet: Removed LNI 192.168.201.5@tcp [ 1968.021066] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1968.172262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 15 [ 1968.242677] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1969.458416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 1969.793692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1970.866735] LNet: Removed LNI 192.168.201.5@tcp [ 1970.888829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1970.941584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1970.977506] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1971.706224] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 128 [ 1971.773203] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1971.816382] LNet: Removed LNI 192.168.201.5@tcp [ 1971.839525] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1971.894564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1971.919473] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1972.603436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer -1 [ 1972.703560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1972.735822] LNet: Removed LNI 192.168.201.5@tcp [ 1972.754633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1972.808545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 128 [ 1972.899785] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1972.953235] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1973.021805] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 1000000000000000000000000 [ 1973.099432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1973.123876] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1973.166599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer foo [ 1973.249431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1973.327424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1973.399577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 8 [ 1973.458406] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1974.472289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1974.497606] LNet: Removed LNI 192.168.201.5@tcp [ 1974.510747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1974.580552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 1983.132352] Lustre: DEBUG MARKER: == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 08:51:24 (1789131084) [ 1983.256470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1983.288218] LNet: 85405:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1983.293450] LNet: 85405:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 9 previous similar messages [ 1983.330281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1983.416199] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 [ 1983.456029] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/180] [ 1985.448572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1985.486914] LNetError: 85550:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1985.495341] LNetError: 85550:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 1985.539340] LNet: Removed LNI 192.168.201.5@tcp231 [ 1985.582979] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1985.713712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1985.759394] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/180] [ 1985.772453] LNet: 85558:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1986.429592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1986.495581] LNet: Removed LNI 192.168.201.5@tcp231 [ 1986.530946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=-1 [ 1986.583806] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/180] [ 1987.717748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1987.754024] LNet: Removed LNI 192.168.201.5@tcp231 [ 1987.770899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1987.870945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1987.905418] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/0] [ 1987.909945] LNet: 85710:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1988.548416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1988.576681] LNet: Removed LNI 192.168.201.5@tcp231 [ 1988.593550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=0 [ 1988.613023] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/0] [ 1989.824466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1989.872883] LNet: Removed LNI 192.168.201.5@tcp231 [ 1989.913801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1990.037312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1990.066115] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/60] [ 1990.072678] LNet: 85862:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1990.750308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1990.788320] LNet: Removed LNI 192.168.201.5@tcp231 [ 1990.813087] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=60 [ 1990.835558] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/60] [ 1991.782335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1991.818727] LNet: Removed LNI 192.168.201.5@tcp231 [ 1991.834352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1991.880303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1991.898277] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/180] [ 1992.386794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1992.413062] LNet: Removed LNI 192.168.201.5@tcp231 [ 1992.428432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=180 [ 1992.451977] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/180] [ 2002.170753] Lustre: DEBUG MARKER: == sanity-lnet test 232: Test setting ToS value ========== 08:51:43 (1789131103) [ 2002.373506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2002.439916] LNet: Removed LNI 192.168.201.5@tcp231 [ 2002.475204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2002.589149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 [ 2002.647453] LNet: Added LNI 192.168.201.5@tcp232 [8/256/0/180] [ 2003.382962] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 2003.430733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 2003.461601] LNet: Removed LNI 192.168.201.5@tcp232 [ 2003.479160] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos -1 [ 2003.508931] LNet: Added LNI 192.168.201.5@tcp232 [8/256/0/180] [ 2004.149873] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 2004.183639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 2004.216182] LNet: Removed LNI 192.168.201.5@tcp232 [ 2004.231425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 104 [ 2004.254619] LNet: Added LNI 192.168.201.5@tcp232 [8/256/0/180] [ 2004.891090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 2004.930756] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 2004.959431] LNet: Removed LNI 192.168.201.5@tcp232 [ 2004.976677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 106 [ 2005.025819] LNet: Added LNI 192.168.201.5@tcp232 [8/256/0/180] [ 2005.822562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 2005.865894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 2005.908467] LNet: Removed LNI 192.168.201.5@tcp232 [ 2005.927490] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 300 [ 2013.972352] Lustre: DEBUG MARKER: == sanity-lnet test 233: Check for successful resends ==== 08:51:55 (1789131115) [ 2015.396725] Key type .llcrypt unregistered [ 2015.398981] Key type ._llcrypt unregistered [ 2023.723898] Key type ._llcrypt registered [ 2023.731398] Key type .llcrypt registered [ 2023.996637] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2024.018637] alg: No test for adler32 (adler32-zlib) [ 2025.010592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2025.050205] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2034.245950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 2045.425356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2045.498475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2045.545487] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 2047.942708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_interrupt -r 2 [ 2048.025711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_interrupt -r 2 [ 2048.108783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_interrupt -r 2 [ 2048.195183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_interrupt -r 2 [ 2048.245663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_interrupt -r 2 [ 2048.327886] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_interrupt -r 2 [ 2048.373662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_interrupt -r 2 [ 2048.429350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_interrupt -r 2 [ 2048.500492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2048.544637] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 2048.548432] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 2048.983778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2049.043384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2049.079570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2049.142760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2049.215180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2049.277718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2049.362592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2049.431325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2049.473675] LNet: 88520:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2049.509750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2049.581705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2049.632963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2049.718835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2049.812234] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2049.874593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2050.638406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_dropped -r 2 [ 2050.700592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_dropped -r 2 [ 2050.791242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_dropped -r 2 [ 2050.868631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_dropped -r 2 [ 2050.900209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_dropped -r 2 [ 2050.954423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_dropped -r 2 [ 2051.048421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_dropped -r 2 [ 2051.128225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_dropped -r 2 [ 2051.244811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2051.276395] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 2051.575263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2051.605772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2051.631801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2051.657832] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2051.710900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2051.749728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2051.774383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2051.802299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2051.827693] LNet: 88640:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2051.840606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2051.898583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2051.944270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2052.001320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2052.095359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2052.181569] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2052.944733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_aborted -r 2 [ 2052.997692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_aborted -r 2 [ 2053.052291] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_aborted -r 2 [ 2053.114049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_aborted -r 2 [ 2053.165671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_aborted -r 2 [ 2053.245502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_aborted -r 2 [ 2053.333438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_aborted -r 2 [ 2053.380298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_aborted -r 2 [ 2053.440108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2053.492064] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 2053.852200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2053.886939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2053.921368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2053.965497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2054.025663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2054.070367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2054.122453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2054.152635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2054.166971] LNet: 88760:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2054.188752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2054.226580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2054.262381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2054.340168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2054.422245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2054.463323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2055.195531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_no_route -r 2 [ 2055.252569] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_no_route -r 2 [ 2055.302690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_no_route -r 2 [ 2055.406812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_no_route -r 2 [ 2055.500660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_no_route -r 2 [ 2055.601361] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_no_route -r 2 [ 2055.688302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_no_route -r 2 [ 2055.758629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_no_route -r 2 [ 2055.824424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2055.869654] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 2056.282424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2056.359255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2056.435531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2056.489242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2056.550948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2056.589864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2056.625618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2056.667150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2056.702416] LNet: 88881:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2056.730615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2056.772432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2056.807335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2056.846175] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2056.883899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2056.924974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2057.889362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_timeout -r 2 [ 2057.962018] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_timeout -r 2 [ 2058.067995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_timeout -r 2 [ 2058.141687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_timeout -r 2 [ 2058.225737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_timeout -r 2 [ 2058.298177] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_timeout -r 2 [ 2058.371970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_timeout -r 2 [ 2058.466891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_timeout -r 2 [ 2058.556742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2058.942790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2059.000862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2059.041411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2059.142547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2059.196083] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2059.277790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2059.356635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2059.433944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2059.518877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2059.575767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2059.639119] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2059.714938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2059.787633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2059.858731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2060.728586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_dropped -r 2 [ 2060.792969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e remote_dropped -r 2 [ 2060.860607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_dropped -r 2 [ 2060.945448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e remote_dropped -r 2 [ 2061.021573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e remote_dropped -r 2 [ 2061.076968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e remote_dropped -r 2 [ 2061.150485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e remote_dropped -r 2 [ 2061.241162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e remote_dropped -r 2 [ 2061.336991] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2061.395901] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 2061.404590] LNet: Skipped 1 previous similar message [ 2061.408185] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.105@tcp [ 2061.813647] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2061.881358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2061.944742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2061.996151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2062.039448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2062.104038] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2062.175354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2062.237576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2062.271755] LNet: 89120:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2062.286045] LNet: 89120:0:(api-ni.c:296:recovery_interval_set()) Skipped 1 previous similar message [ 2062.318489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2062.370445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2062.460354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2062.521643] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2062.586850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2062.643830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2069.320473] LNet: 89199:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2069.331718] LNetError: 89199:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2069.341856] LNet: Removed LNI 192.168.201.5@tcp [ 2069.350790] LNet: Removed LNI 192.168.201.5@tcp1 [ 2069.775984] Key type .llcrypt unregistered [ 2069.779669] Key type ._llcrypt unregistered [ 2092.530902] Key type ._llcrypt registered [ 2092.532730] Key type .llcrypt registered [ 2093.984664] Lustre: DEBUG MARKER: == sanity-lnet test 234: Verify NI status changes with manual health value transitions ========================================================== 08:53:15 (1789131195) [ 2094.742312] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2094.759481] alg: No test for adler32 (adler32-zlib) [ 2095.738703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2095.792068] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2095.815158] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2096.645302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 0 [ 2096.677182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.5@tcp [ 2096.696597] LNetError: 90181:0:(api-ni.c:4473:lnet_ni_set_healthv()) manually adding local NI 192.168.201.5@tcp to recovery [ 2096.764539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --nid 192.168.201.5@tcp [ 2096.873798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2104.606029] Lustre: DEBUG MARKER: == sanity-lnet test 235: Verify NI status DOWN when health reaches 0 after failures ========================================================== 08:53:26 (1789131206) [ 2104.729609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2104.772261] LNet: 90767:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2104.788584] LNetError: 90767:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2104.832336] LNet: Removed LNI 192.168.201.5@tcp [ 2104.865258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2104.925988] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2104.950740] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2105.770449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2105.813573] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 2106.786653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 100 --nid 192.168.201.5@tcp [ 2106.830301] LNetError: 91037:0:(api-ni.c:4473:lnet_ni_set_healthv()) manually adding local NI 192.168.201.5@tcp to recovery [ 2106.976530] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 2106.985881] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 2108.002434] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 2108.109759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 900 --nid 192.168.201.5@tcp [ 2120.537084] Lustre: DEBUG MARKER: == sanity-lnet test 236: Local NI state propagates to routers ========================================================== 08:53:41 (1789131221) [ 2123.029439] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_236 Need at least 2 remote nodes found oleg105-server [ 2125.623225] Lustre: DEBUG MARKER: == sanity-lnet test 237: Check MR forwarding feature ===== 08:53:46 (1789131226) [ 2128.030251] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_237 Need at least 2 remote nodes found oleg105-server [ 2130.138314] Lustre: DEBUG MARKER: == sanity-lnet test 238: Check MR forwarding feature skipped for non-MR source ========================================================== 08:53:51 (1789131231) [ 2132.451493] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_238 Need at least 2 remote nodes found oleg105-server [ 2134.555835] Lustre: DEBUG MARKER: == sanity-lnet test 241: Check setting LND timeout value via lnetctl updates tunables ========================================================== 08:53:56 (1789131236) [ 2134.890790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2134.922168] LNet: 92684:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2134.937788] LNetError: 92684:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2134.947668] LNet: Removed LNI 192.168.201.5@tcp [ 2134.958251] LNet: Removed LNI 192.168.201.5@tcp1 [ 2135.011652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2135.117742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2135.141877] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2136.051060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set -h [ 2136.131487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 51 [ 2136.192702] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 2136.304696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 0 [ 2136.362320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 2136.412136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 50 [ 2143.683543] Lustre: DEBUG MARKER: == sanity-lnet test 245: Check read-only params do not appear in export --backup ========================================================== 08:54:05 (1789131245) [ 2143.854485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2143.922578] LNet: 93404:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2143.945128] LNetError: 93404:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2143.953955] LNetError: 93404:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2144.025511] LNet: Removed LNI 192.168.201.5@tcp [ 2144.135568] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2144.194921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2144.213130] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2152.724539] Lustre: DEBUG MARKER: == sanity-lnet test 250: test that linux routes are added ========================================================== 08:54:14 (1789131254) [ 2152.824280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2152.858962] LNet: 94104:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2152.868834] LNetError: 94104:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2152.896079] LNet: Removed LNI 192.168.201.5@tcp [ 2152.922795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2153.009638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2153.045258] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2161.428689] Lustre: DEBUG MARKER: == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 08:54:23 (1789131263) [ 2163.540887] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_251 Need kfi NETTYPE [ 2165.701425] Lustre: DEBUG MARKER: == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 08:54:27 (1789131267) [ 2166.430983] LNet: 95149:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2166.449377] LNetError: 95149:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2166.480884] LNet: Removed LNI 192.168.201.5@tcp [ 2167.426751] Key type .llcrypt unregistered [ 2167.428188] Key type ._llcrypt unregistered [ 2174.405194] Key type ._llcrypt registered [ 2174.408410] Key type .llcrypt registered [ 2174.642246] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2174.675821] alg: No test for adler32 (adler32-zlib) [ 2175.685368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2175.752347] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2185.410283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 2191.241565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --timeout 15 192.168.201.105@tcp [ 2191.780362] LNet: 95575:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2191.789578] LNetError: 95575:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2191.811516] LNet: Removed LNI 192.168.201.5@tcp [ 2192.345510] Key type .llcrypt unregistered [ 2192.347079] Key type ._llcrypt unregistered [ 2205.857970] Key type ._llcrypt registered [ 2205.861342] Key type .llcrypt registered [ 2207.266590] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253 [ 2209.173963] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254 [ 2211.049415] Lustre: DEBUG MARKER: == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 08:55:12 (1789131312) [ 2212.546143] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_255 kernel ip2nets deprecated in 2.17.53 [ 2214.591829] Lustre: DEBUG MARKER: == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 08:55:16 (1789131316) [ 2216.652385] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_256 Need at least 2 remote nodes found oleg105-server [ 2218.673602] Lustre: DEBUG MARKER: == sanity-lnet test 257: Test graceful router shutdown/startup ========================================================== 08:55:20 (1789131320) [ 2220.800229] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_257 Need at least 3 remote nodes found oleg105-server [ 2222.806455] Lustre: DEBUG MARKER: == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 08:55:24 (1789131324) [ 2225.136148] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2225.334439] alg: No test for adler32 (adler32-zlib) [ 2226.570740] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 2226.905316] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2228.586237] Key type lgssc registered [ 2230.251845] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2255.609487] Key type lgssc unregistered [ 2256.036299] LNet: 98754:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2256.045410] LNetError: 98754:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2256.091624] LNet: Removed LNI 192.168.201.5@tcp [ 2257.240812] Key type .llcrypt unregistered [ 2257.245990] Key type ._llcrypt unregistered [ 2278.201167] Key type ._llcrypt registered [ 2278.203745] Key type .llcrypt registered [ 2279.880713] Lustre: DEBUG MARKER: == sanity-lnet test 265: Import of invalid CPT should fail ========================================================== 08:56:21 (1789131381) [ 2280.539548] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2280.590232] alg: No test for adler32 (adler32-zlib) [ 2281.526973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2281.590430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2281.634198] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2281.712216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2281.778637] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2289.343860] Lustre: DEBUG MARKER: == sanity-lnet test 266: Validate CPT parsing in network sequence ========================================================== 08:56:31 (1789131391) [ 2289.589703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2289.625876] LNet: 100170:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2289.655045] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2289.723821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2289.758056] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2290.922586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2290.955818] LNet: 100314:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2290.968950] LNetError: 100314:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2290.980305] LNet: Removed LNI 192.168.201.5@tcp [ 2291.000335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2291.058783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 2291.105365] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2291.672947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2291.702642] LNetError: 100342:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2291.715033] LNet: Removed LNI 192.168.201.5@tcp [ 2291.731878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2291.791386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 2291.828704] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2299.594347] Lustre: DEBUG MARKER: == sanity-lnet test 270: Test NID metadata =============== 08:56:41 (1789131401) [ 2299.738889] LNet: 100924:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2299.747521] LNet: 100924:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 2299.753725] LNetError: 100924:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2299.783364] LNet: Removed LNI 192.168.201.5@tcp [ 2300.717631] Key type .llcrypt unregistered [ 2300.718842] Key type ._llcrypt unregistered [ 2309.069750] Key type ._llcrypt registered [ 2309.074610] Key type .llcrypt registered [ 2309.362895] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2309.389530] alg: No test for adler32 (adler32-zlib) [ 2310.395822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2310.469417] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2310.471475] LNet: *** cfs_fail_loc=e003, val=100*** [ 2311.592325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2311.613034] LNet: *** cfs_fail_loc=e003, val=100*** [ 2311.615492] LNet: Skipped 2 previous similar messages [ 2311.624032] Ping reply received with mapping: [ 2311.628087] found metadata reply NI[192.168.201.5@tcp] [ 2311.635521] 05 c9 a8 c0 00 00 02 00 5a 5a 5a 5a 5a 5a 5a 5a [ 2311.643637] 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a [ 2311.650583] 5a 5a 5a 5a 5a 5a 5a 5a [ 2311.815505] LNet: 101335:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2311.825279] LNetError: 101335:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2312.886735] LNet: Removed LNI 192.168.201.5@tcp [ 2313.607182] Key type .llcrypt unregistered [ 2313.615695] Key type ._llcrypt unregistered [ 2327.153208] Key type ._llcrypt registered [ 2327.156464] Key type .llcrypt registered [ 2328.938507] Lustre: DEBUG MARKER: == sanity-lnet test 280: Don't panic when request_module fails ========================================================== 08:57:10 (1789131430) [ 2329.554366] Key type .llcrypt unregistered [ 2329.556424] Key type ._llcrypt unregistered [ 2337.813383] Key type ._llcrypt registered [ 2337.815879] Key type .llcrypt registered [ 2338.048521] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2338.066498] alg: No test for adler32 (adler32-zlib) [ 2339.629736] LNetError: 102393:0:(api-ni.c:2850:lnet_load_lnd()) Can't load LND tcp, module ksocklnd, rc=256 [ 2340.143421] Key type .llcrypt unregistered [ 2340.145941] Key type ._llcrypt unregistered [ 2346.251951] Key type ._llcrypt registered [ 2346.254062] Key type .llcrypt registered [ 2347.519751] Lustre: DEBUG MARKER: == sanity-lnet test 290: Check that lnet acceptor is using explicit address binding ========================================================== 08:57:29 (1789131449) [ 2348.547361] Key type .llcrypt unregistered [ 2348.550221] Key type ._llcrypt unregistered [ 2358.044932] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2358.063827] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2360.954993] Key type ._llcrypt registered [ 2360.960283] Key type .llcrypt registered [ 2361.259327] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2361.290606] alg: No test for adler32 (adler32-zlib) [ 2362.377878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2362.549842] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2377.408486] Lustre: DEBUG MARKER: == sanity-lnet test 291: load lnet using custom accept port for bulk tcp connections ========================================================== 08:57:59 (1789131479) [ 2377.585991] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2377.637812] LNet: 104177:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2377.657291] LNetError: 104177:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2377.692798] LNet: Removed LNI 10.1.2.3@tcp [ 2377.740812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2379.073847] Key type .llcrypt unregistered [ 2379.081473] Key type ._llcrypt unregistered [ 2388.788161] Key type ._llcrypt registered [ 2388.789668] Key type .llcrypt registered [ 2389.268487] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2389.326897] alg: No test for adler32 (adler32-zlib) [ 2390.367847] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2390.475653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2390.709542] LNet: 104476:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2390.715265] LNetError: 104476:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2390.739662] LNet: Removed LNI 192.168.201.5@tcp [ 2391.674141] Key type .llcrypt unregistered [ 2391.676160] Key type ._llcrypt unregistered [ 2406.250842] Key type ._llcrypt registered [ 2406.252986] Key type .llcrypt registered [ 2407.969949] Lustre: DEBUG MARKER: == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 08:58:29 (1789131509) [ 2408.744419] Key type .llcrypt unregistered [ 2408.748240] Key type ._llcrypt unregistered [ 2416.684746] Key type ._llcrypt registered [ 2416.687229] Key type .llcrypt registered [ 2417.168519] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2417.197100] alg: No test for adler32 (adler32-zlib) [ 2425.210826] Lustre: DEBUG MARKER: == sanity-lnet test 301: Check for dynamic adds of same/wrong interface (memory leak) ========================================================== 08:58:46 (1789131526) [ 2425.313510] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2425.383500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2425.492505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2425.513123] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2426.063341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2426.092779] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if ens2 [ 2426.121859] LNet: Removed LNI 192.168.201.5@tcp [ 2426.399976] LNet: 106234:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2426.746513] Key type .llcrypt unregistered [ 2426.748330] Key type ._llcrypt unregistered [ 2440.939260] Key type ._llcrypt registered [ 2440.943081] Key type .llcrypt registered [ 2442.509957] Lustre: DEBUG MARKER: == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 08:59:03 (1789131543) [ 2443.135432] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2443.150951] alg: No test for adler32 (adler32-zlib) [ 2444.054027] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2444.118916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2444.143627] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2444.942513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2444.982960] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug peer --nid 192.168.201.5@tcp [ 2445.013395] LNet: Conn 0 [type, closing, crefcnt, srefcnt]: 2, 0, 1, 1 [ 2445.018395] LNet: Conn 0 rx [scheduled, ready, state]: 0, 0, 1 [ 2445.021368] LNet: Conn 0 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2444, 0, 0 [ 2445.026755] LNet: Conn 0 sched [nconns, cpt]: 4, 0 [ 2445.034472] LNet: Conn 1 [type, closing, crefcnt, srefcnt]: 3, 0, 1, 1 [ 2445.038604] LNet: Conn 1 rx [scheduled, ready, state]: 0, 0, 1 [ 2445.042913] LNet: Conn 1 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2444, 0, 0 [ 2445.048816] LNet: Conn 1 sched [nconns, cpt]: 4, 0 [ 2445.052904] LNet: Conn 2 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 2445.057353] LNet: Conn 2 rx [scheduled, ready, state]: 0, 0, 1 [ 2445.060618] LNet: Conn 2 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2444, 0, 2494 [ 2445.064707] LNet: Conn 2 sched [nconns, cpt]: 4, 0 [ 2445.067139] LNet: Conn 3 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 2445.068813] LNet: Conn 3 rx [scheduled, ready, state]: 0, 0, 1 [ 2445.073299] LNet: Conn 3 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2444, 0, 2494 [ 2445.079269] LNet: Conn 3 sched [nconns, cpt]: 4, 0 [ 2452.875635] Lustre: DEBUG MARKER: == sanity-lnet test 303: Check peer NI health after link down ========================================================== 08:59:14 (1789131554) [ 2453.435163] LNet: 107777:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2453.441383] LNetError: 107777:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2453.474302] LNet: Removed LNI 192.168.201.5@tcp [ 2454.471906] Key type .llcrypt unregistered [ 2454.476390] Key type ._llcrypt unregistered [ 2462.275274] Key type ._llcrypt registered [ 2462.281811] Key type .llcrypt registered [ 2462.516820] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2462.534801] alg: No test for adler32 (adler32-zlib) [ 2463.502755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2463.575265] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2472.273691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 2484.024650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2484.092659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2484.139553] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 2485.481901] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 2485.705145] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2485.713760] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2485.717956] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 2487.121123] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp99 --if test1pg [ 2487.179402] LNet: Added LNI 10.1.2.3@tcp99 [8/256/0/180] [ 2501.949885] Lustre: DEBUG MARKER: == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 09:00:03 (1789131603) [ 2502.080812] LNet: 109276:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2502.096516] LNetError: 109276:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2502.106947] LNet: Removed LNI 192.168.201.5@tcp [ 2503.136279] LNet: Removed LNI 192.168.201.5@tcp1 [ 2503.139854] LNetError: 109276:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2503.149742] LNetError: 109276:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2503.168291] LNet: Removed LNI 10.1.2.3@tcp99 [ 2503.958623] Key type .llcrypt unregistered [ 2503.961619] Key type ._llcrypt unregistered [ 2511.302234] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2511.307617] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2514.150354] Key type ._llcrypt registered [ 2514.155753] Key type .llcrypt registered [ 2514.515964] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2514.552099] alg: No test for adler32 (adler32-zlib) [ 2515.634776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2515.800580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2515.837505] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2517.033625] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2517.085934] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2520.181845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.1.2.3@tcp --lock_prim [ 2520.228448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2520.258885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 10.1.2.3@tcp --state 1048577 [ 2520.276412] LNetError: 110033:0:(peer.c:2362:lnet_del_peer_ni()) peer 10.1.2.3@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 2520.312967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2520.345617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2520.765763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2520.843306] LNet: 110053:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2520.852708] LNetError: 110053:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2520.862914] LNet: Removed LNI 10.1.2.3@tcp [ 2521.898707] LNet: Removed LNI 192.168.201.5@tcp [ 2523.046545] Key type .llcrypt unregistered [ 2523.052585] Key type ._llcrypt unregistered [ 2538.653393] Key type ._llcrypt registered [ 2538.656309] Key type .llcrypt registered [ 2540.198695] Lustre: DEBUG MARKER: == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 09:00:41 (1789131641) [ 2540.776129] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2540.794615] alg: No test for adler32 (adler32-zlib) [ 2541.827056] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2541.925612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2541.990083] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2542.750126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2542.858530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping oleg105-client@tcp [ 2549.527805] Lustre: DEBUG MARKER: == sanity-lnet test 310: Set timeout and verify ========== 09:00:51 (1789131651) [ 2549.683536] LNet: 111586:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2549.690778] LNetError: 111586:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2549.719206] LNet: Removed LNI 192.168.201.5@tcp [ 2550.666855] Key type .llcrypt unregistered [ 2550.673828] Key type ._llcrypt unregistered [ 2557.275773] Key type ._llcrypt registered [ 2557.279873] Key type .llcrypt registered [ 2557.543627] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2557.574479] alg: No test for adler32 (adler32-zlib) [ 2558.625269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure --all [ 2558.719598] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2558.815576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 2566.281485] Lustre: DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 09:01:08 (1789131668) [ 2568.294452] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type [ 2570.212742] Lustre: DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 09:01:12 (1789131672) [ 2572.177904] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type [ 2574.121952] Lustre: DEBUG MARKER: == sanity-lnet test 350: Check import --exec of ping/discover ========================================================== 09:01:15 (1789131675) [ 2574.346745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2574.390487] LNet: 113030:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2574.407677] LNetError: 113030:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2574.418966] LNet: Removed LNI 192.168.201.5@tcp [ 2574.436721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2574.482612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2574.497715] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2575.125766] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 2575.158548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 2582.422958] Lustre: DEBUG MARKER: == sanity-lnet test 400: Check for udsp add/delete net rule without net num ========================================================== 09:01:23 (1789131683) [ 2582.555948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2582.584894] LNet: 113738:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2582.592440] LNetError: 113738:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2582.610760] LNet: Removed LNI 192.168.201.5@tcp [ 2582.635723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2582.705037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --priority 0 [ 2582.735260] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2583.360183] Key type .llcrypt unregistered [ 2583.361520] Key type ._llcrypt unregistered [ 2595.597538] Key type ._llcrypt registered [ 2595.599981] Key type .llcrypt registered [ 2597.256253] Lustre: DEBUG MARKER: == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 09:01:38 (1789131698) [ 2597.768498] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2597.788645] alg: No test for adler32 (adler32-zlib) [ 2598.759780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2598.865986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2598.903236] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2599.819750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2599.888443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2609.128121] Lustre: DEBUG MARKER: == sanity-lnet test 402: Destination net rule should not panic ========================================================== 09:01:50 (1789131710) [ 2609.212401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2609.275727] LNet: 115294:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2609.297698] LNetError: 115294:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2609.318193] LNet: Removed LNI 192.168.201.5@tcp [ 2609.390597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2609.487203] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst kfi --priority 0 [ 2609.517907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 402@kfi [ 2617.673827] Lustre: DEBUG MARKER: == sanity-lnet test 403: Add and delete udsp net and nid rules ========================================================== 09:01:58 (1789131718) [ 2617.851547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2617.886604] LNet: 115874:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2617.907943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2618.000067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2618.587943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2618.644458] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2618.776660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src 1.1.1.1@tcp --prio 1 [ 2619.381991] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2619.445277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2619.528680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2619.552745] LNet: 115914:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2619.589178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2619.690581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2620.292784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2620.329664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2620.423485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst 1.1.1.1@tcp --prio 1 [ 2621.002359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2621.031650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2628.426984] Lustre: DEBUG MARKER: == sanity-lnet test 404: Check udsp src/dst net rule ===== 09:02:10 (1789131730) [ 2628.572267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2628.591276] LNet: 116509:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2628.621386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2628.742500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2628.761982] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2629.330145] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2629.343945] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2630.468177] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2630.550802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2630.904900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2630.976333] LNetError: 116794:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2632.035603] LNet: Removed LNI 192.168.201.5@tcp [ 2632.042937] LNetError: 116794:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2632.062836] LNet: Removed LNI 192.168.201.5@tcp2 [ 2632.113698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2632.198514] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2632.212488] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2632.222134] LNet: 116800:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2632.693607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2632.862766] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2632.874063] LNet: 116837:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2632.876842] LNet: 116837:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 2633.888578] LNet: Removed LNI 192.168.201.5@tcp [ 2633.896789] LNet: Removed LNI 192.168.201.5@tcp2 [ 2633.918725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2633.999323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2634.035291] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2634.790305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2634.810116] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2635.788780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2635.827640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 2635.889384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2636.146148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2636.190232] LNetError: 117122:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2636.197415] LNetError: 117122:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 2636.207599] LNet: Removed LNI 192.168.201.5@tcp [ 2637.225877] LNet: Removed LNI 192.168.201.5@tcp2 [ 2637.329906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2637.459551] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2637.467144] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2637.473714] LNet: 117128:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2638.354228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2638.533307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2638.548393] LNet: Removed LNI 192.168.201.5@tcp [ 2639.592266] LNet: Removed LNI 192.168.201.5@tcp2 [ 2639.683190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2639.815864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2639.880735] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2641.002928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2641.026034] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2642.325932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2642.410764] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2642.683405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2642.698467] LNet: 117445:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2642.702644] LNet: 117445:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 2642.707563] LNetError: 117445:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2642.711098] LNetError: 117445:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 2643.744509] LNet: Removed LNI 192.168.201.5@tcp [ 2643.765955] LNet: Removed LNI 192.168.201.5@tcp2 [ 2643.843905] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2644.031144] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2644.054947] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2644.085861] LNet: 117452:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2645.359877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2645.651889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2646.688139] LNet: Removed LNI 192.168.201.5@tcp [ 2646.700379] LNet: Removed LNI 192.168.201.5@tcp2 [ 2646.845876] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2646.984417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2647.018627] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2648.674815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2648.729201] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2650.083379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2650.127581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 2650.182571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2650.484262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2651.552183] LNet: Removed LNI 192.168.201.5@tcp [ 2651.553758] LNetError: 117774:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2651.566266] LNetError: 117774:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 4 previous similar messages [ 2651.612986] LNet: Removed LNI 192.168.201.5@tcp2 [ 2651.696979] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2651.851371] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2651.865043] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2651.876460] LNet: 117780:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2653.257541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2653.715492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2653.763834] LNet: Removed LNI 192.168.201.5@tcp [ 2654.822766] LNet: Removed LNI 192.168.201.5@tcp2 [ 2654.881483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2655.071342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2655.146273] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2656.270903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2656.317306] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2657.407431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2657.445289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2657.505171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2658.079174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2659.168167] LNet: Removed LNI 192.168.201.5@tcp [ 2659.190288] LNet: Removed LNI 192.168.201.5@tcp2 [ 2659.269629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2659.425859] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2659.434110] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2659.442494] LNet: 118122:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2660.321524] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2661.103120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2661.137706] LNet: 118174:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2661.144525] LNet: 118174:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 4 previous similar messages [ 2662.176333] LNet: Removed LNI 192.168.201.5@tcp [ 2662.197695] LNet: Removed LNI 192.168.201.5@tcp2 [ 2662.247363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2662.385181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2662.437757] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2663.721719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2663.754241] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2665.286922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2665.378972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 2665.447668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 2665.569559] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2666.118770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2667.232388] LNet: Removed LNI 192.168.201.5@tcp [ 2668.330704] LNet: Removed LNI 192.168.201.5@tcp2 [ 2668.429438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2668.618976] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2668.634061] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2668.648568] LNet: 118482:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2669.857600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2679.354583] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_410 skipping ALWAYS excluded test 410 [ 2681.771902] Lustre: DEBUG MARKER: == sanity-lnet test 450: Check import of multiple interfaces ========================================================== 09:03:02 (1789131782) [ 2681.934405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2681.993842] LNetError: 119236:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2682.005876] LNetError: 119236:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 8 previous similar messages [ 2682.049881] LNet: Removed LNI 192.168.201.5@tcp [ 2683.118735] LNet: Removed LNI 192.168.201.5@tcp2 [ 2683.168967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2683.524475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2683.546482] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2684.363444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2684.391953] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2685.767288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2685.783653] LNet: Removed LNI 10.1.2.3@tcp [ 2685.794553] LNet: Removed LNI 192.168.201.5@tcp [ 2685.821213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2685.873995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-450.yaml [ 2685.892933] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2685.897139] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2686.203104] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del /tmp/sanity-lnet-450.yaml [ 2686.240838] LNet: Removed LNI 192.168.201.5@tcp [ 2686.258898] LNet: Removed LNI 10.1.2.3@tcp [ 2693.315948] Lustre: DEBUG MARKER: == sanity-lnet test 475: per-operation latency stats populate and reset ========================================================== 09:03:14 (1789131794) [ 2693.481791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2693.512069] LNet: 120108:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2693.522961] LNet: 120108:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 2693.600163] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2693.745299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2693.802448] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2694.745332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 2694.794248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2694.828871] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2694.870920] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2694.910493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2694.972445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2695.020508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2695.063243] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2695.114253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2695.177705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2695.216624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2695.401964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 2704.033404] Lustre: DEBUG MARKER: == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 09:03:25 (1789131805) [ 2704.132692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2705.254945] LNet: Removed LNI 192.168.201.5@tcp [ 2705.318964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2705.553506] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 2705.883231] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2705.891391] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2705.894177] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 2708.371476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2708.394548] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2709.828919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2709.889550] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2714.482544] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2714.565698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d *@tcp -r 1 -l 1 -m PUT [ 2714.763389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 192.168.201.5@tcp -m PUT -e local_timeout -r 1 [ 2714.978636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 10.1.2.3@tcp -m PUT -e local_timeout -r 1 [ 2715.148946] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2715.155975] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2715.807559] LNetError: 120917:0:(lib-move.c:4801:lnet_parse()) 192.168.201.5@tcp, src 192.168.201.5@tcp: Dropping PUT (error -108 looking up sender) [ 2716.779208] LNetError: 121379:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2716.790661] LNetError: 121379:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 4 previous similar messages [ 2716.812867] LNet: Removed LNI 10.1.2.3@tcp [ 2717.869489] LNet: Removed LNI 192.168.201.5@tcp [ 2718.936130] Key type .llcrypt unregistered [ 2718.950501] Key type ._llcrypt unregistered [ 2736.899308] Key type ._llcrypt registered [ 2736.908519] Key type .llcrypt registered [ 2738.711802] Lustre: DEBUG MARKER: == sanity-lnet test 501: Verify lnetctl peer set --health (SR) ========================================================== 09:03:59 (1789131839) [ 2739.463444] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2739.492042] alg: No test for adler32 (adler32-zlib) [ 2740.469952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2740.840906] LNet: 122234:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2741.377964] Key type .llcrypt unregistered [ 2741.383695] Key type ._llcrypt unregistered [ 2749.804995] Key type ._llcrypt registered [ 2749.809364] Key type .llcrypt registered [ 2750.150581] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2750.181038] alg: No test for adler32 (adler32-zlib) [ 2751.180885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2751.213683] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2759.105559] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 2759.363165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 2760.064782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.105@tcp [ 2760.421314] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.105@tcp [ 2760.463432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2760.510405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2760.560394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2760.608580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2760.650501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2760.713331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2760.775337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2760.835938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2760.879452] LNet: 122688:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2760.924300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2760.999158] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2761.075457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2761.136926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2761.212380] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2761.280412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2762.263134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 2762.657198] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2762.700943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2762.732646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2762.784799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2762.825702] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2762.867700] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2762.893302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2762.926995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2762.954100] LNet: 122806:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2762.980225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2763.013315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2763.059407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2763.115873] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2763.157396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2763.202066] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2764.072820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.105@tcp [ 2764.386808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2764.427188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2764.460614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2764.504438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2764.563578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2764.605973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2764.649925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2764.694425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2764.719641] LNet: 122923:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2764.763795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2764.807452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2764.866651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2764.897213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2764.943669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2764.983593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2765.737654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 2766.021775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2766.057216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2766.091573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2766.123636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2766.154587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2766.186804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2766.234921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2766.290473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2766.344493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2766.395240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2766.445101] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2766.515840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2766.578224] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2766.630265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2767.166915] LNet: 123091:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2767.174938] LNetError: 123091:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2767.201477] LNet: Removed LNI 192.168.201.5@tcp [ 2767.814196] Key type .llcrypt unregistered [ 2767.815953] Key type ._llcrypt unregistered [ 2791.958578] Key type ._llcrypt registered [ 2791.969757] Key type .llcrypt registered [ 2793.961811] Lustre: DEBUG MARKER: == sanity-lnet test 502: Verify lnetctl peer set --health (MR) ========================================================== 09:04:55 (1789131895) [ 2794.617885] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2794.634600] alg: No test for adler32 (adler32-zlib) [ 2795.769641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2796.503451] LNet: 123981:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2797.783685] Key type .llcrypt unregistered [ 2797.785677] Key type ._llcrypt unregistered [ 2806.190788] Key type ._llcrypt registered [ 2806.194724] Key type .llcrypt registered [ 2806.551001] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2806.575605] alg: No test for adler32 (adler32-zlib) [ 2807.653917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2807.719884] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2817.037643] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 2827.270735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2827.332590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2827.365767] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 2828.554498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 2829.365626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.105@tcp [ 2829.744419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2829.783845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2829.828429] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2829.889060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2829.968039] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2830.040439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2830.095873] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2830.132828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2830.152483] LNet: 124639:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2830.193117] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2830.266108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2830.306022] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.105@tcp [ 2830.354096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2830.460838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2830.581353] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2830.657518] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2831.650633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 2831.987336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2832.037373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2832.077640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2832.119497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2832.165769] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2832.197463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2832.229982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2832.299170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2832.319613] LNet: 124756:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2832.350335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2832.400200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2832.443225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2832.503384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2832.578087] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2832.630879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2833.308269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.105@tcp [ 2833.615330] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2833.642388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2833.671711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2833.707034] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2833.738565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2833.811963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2833.880612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2833.939395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2833.964419] LNet: 124874:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2833.979389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2834.014621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2834.045081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2834.078308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2834.121954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2834.154811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2834.953654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 2835.338160] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2835.398299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2835.455763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2835.550658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2835.645391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2835.741310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2835.855306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2835.953061] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2836.015347] LNet: 124991:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2836.071222] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2836.151906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2836.258087] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2836.337656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2836.452633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2836.506543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2842.492274] LNet: 125076:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2842.501274] LNetError: 125076:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2843.552297] LNet: Removed LNI 192.168.201.5@tcp [ 2843.561207] LNetError: 125076:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2843.579777] LNet: Removed LNI 192.168.201.5@tcp1 [ 2844.094495] Key type .llcrypt unregistered [ 2844.096860] Key type ._llcrypt unregistered [ 2863.029665] Key type ._llcrypt registered [ 2863.032296] Key type .llcrypt registered [ 2864.404353] Lustre: DEBUG MARKER: == sanity-lnet test 525: keep reachable peer on local net delete ========================================================== 09:06:06 (1789131966) [ 2864.839746] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2864.855818] alg: No test for adler32 (adler32-zlib) [ 2866.007315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2866.106681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2866.133806] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2867.118694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2867.129554] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2867.784993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2867.909138] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp [ 2868.960371] LNet: Removed LNI 192.168.201.5@tcp [ 2877.252155] Lustre: DEBUG MARKER: == sanity-lnet test 550: DD-off: keep route up when gw reachable via another local NI ========================================================== 09:06:19 (1789131979) [ 2879.349985] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_550 Need at least 2 local interfaces for multi-rail [ 2881.174350] Lustre: DEBUG MARKER: == sanity-lnet test 600: lnet_legacy2yaml: basic ip2nets conversion ========================================================== 09:06:23 (1789131983) [ 2890.700417] Lustre: DEBUG MARKER: == sanity-lnet test 601: lnet_legacy2yaml: basic routes conversion ========================================================== 09:06:31 (1789131991) [ 2900.750736] Lustre: DEBUG MARKER: == sanity-lnet test 602: lnet_legacy2yaml: routes with priority ========================================================== 09:06:42 (1789132002) [ 2910.420829] Lustre: DEBUG MARKER: == sanity-lnet test 603: lnet_legacy2yaml: default hop flag ========================================================== 09:06:51 (1789132011) [ 2920.973435] Lustre: DEBUG MARKER: == sanity-lnet test 604: lnet_legacy2yaml: default hop with explicit value ========================================================== 09:07:02 (1789132022) [ 2930.062761] Lustre: DEBUG MARKER: == sanity-lnet test 605: lnet_legacy2yaml: quote leading star in IP pattern ========================================================== 09:07:12 (1789132032) [ 2937.362214] Lustre: DEBUG MARKER: == sanity-lnet test 606: lnet_legacy2yaml: file inputs === 09:07:19 (1789132039) [ 2945.036738] Lustre: DEBUG MARKER: == sanity-lnet test 607: lnet_legacy2yaml: options line with combined ip2nets and routes ========================================================== 09:07:26 (1789132046) [ 2953.227487] Lustre: DEBUG MARKER: == sanity-lnet test 608: lnet_legacy2yaml: error on invalid rule (missing IP pattern) ========================================================== 09:07:35 (1789132055) [ 2961.741558] Lustre: DEBUG MARKER: == sanity-lnet test 609: lnet_legacy2yaml: error on no input ========================================================== 09:07:43 (1789132063) [ 2969.508368] Lustre: DEBUG MARKER: == sanity-lnet test 610: lnet_legacy2yaml: multiple files and inline, check order ========================================================== 09:07:51 (1789132071) [ 2978.625382] Lustre: DEBUG MARKER: == sanity-lnet test 611: lnet_legacy2yaml: invalid hopcount ========================================================== 09:08:00 (1789132080) [ 2987.613723] Lustre: DEBUG MARKER: == sanity-lnet test 612: lnet_legacy2yaml: malformed IP pattern ========================================================== 09:08:08 (1789132088) [ 2996.722457] Lustre: DEBUG MARKER: == sanity-lnet test 613: lnet_legacy2yaml: multiple IP patterns in single rule ========================================================== 09:08:18 (1789132098) [ 3008.440208] Lustre: DEBUG MARKER: == sanity-lnet test 614: lnet_legacy2yaml: interfaces enumeration with multiple IP ranges ========================================================== 09:08:29 (1789132109) [ 3019.672484] Lustre: DEBUG MARKER: == sanity-lnet test 615: lnet_legacy2yaml: modprobe configuration file input ========================================================== 09:08:40 (1789132120) [ 3029.108981] Lustre: DEBUG MARKER: == sanity-lnet test 616: lnet_legacy2yaml: combining modprobe file with explicit ip2nets file ========================================================== 09:08:50 (1789132130) [ 3035.896914] Lustre: DEBUG MARKER: == sanity-lnet test 617: lnet_legacy2yaml: round-trip tunables from LNet to modprobe to YAML ========================================================== 09:08:57 (1789132137) [ 3036.061861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 3036.089112] LNet: 136774:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3036.097642] LNetError: 136774:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3036.119331] LNet: Removed LNI 192.168.201.5@tcp2 [ 3036.164082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 3036.298313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 3036.330403] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 3037.680930] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 3037.741082] LNet: 136919:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3037.758792] LNetError: 136919:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3037.778948] LNet: Removed LNI 192.168.201.5@tcp [ 3037.814403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 3037.884719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 3037.904994] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 3046.849067] Lustre: DEBUG MARKER: == sanity-lnet test 618: lnet_legacy2yaml: alternative route format ========================================================== 09:09:07 (1789132147) [ 3056.540511] Lustre: DEBUG MARKER: == sanity-lnet test 619: lnet_legacy2yaml: multiple alternative format routes from file ========================================================== 09:09:18 (1789132158) [ 3065.743545] Lustre: DEBUG MARKER: == sanity-lnet test 620: lnet_legacy2yaml: mixed legacy and alternative route formats ========================================================== 09:09:27 (1789132167) [ 3076.569707] Lustre: DEBUG MARKER: == sanity-lnet test 621: lnet_legacy2yaml: alternative format with invalid hopcount ========================================================== 09:09:38 (1789132178) [ 3084.213981] Lustre: DEBUG MARKER: == sanity-lnet test 622: lnet_legacy2yaml: alternative format with default-hop flag ========================================================== 09:09:45 (1789132185) [ 3092.892320] Lustre: DEBUG MARKER: == sanity-lnet test 623: lnet_legacy2yaml: basic networks conversion ========================================================== 09:09:54 (1789132194) [ 3101.645163] Lustre: DEBUG MARKER: == sanity-lnet test 624: lnet_legacy2yaml: networks with interfaces ========================================================== 09:10:03 (1789132203) [ 3110.725735] Lustre: DEBUG MARKER: == sanity-lnet test 625: lnet_legacy2yaml: multiple networks ========================================================== 09:10:12 (1789132212) [ 3119.630279] Lustre: DEBUG MARKER: == sanity-lnet test 626: lnet_legacy2yaml: mixed networks, ip2nets, and routes ========================================================== 09:10:21 (1789132221) [ 3128.576673] Lustre: DEBUG MARKER: == sanity-lnet test 627: lnet_legacy2yaml: invalid networks format ========================================================== 09:10:30 (1789132230) [ 3139.098881] Lustre: DEBUG MARKER: == sanity-lnet test 628: lnet_legacy2yaml: missing delimiter error ========================================================== 09:10:39 (1789132239) [ 3148.928325] Lustre: DEBUG MARKER: == sanity-lnet test 629: lnet_legacy2yaml: multiple gateways error ========================================================== 09:10:50 (1789132250) [ 3158.719885] Lustre: DEBUG MARKER: == sanity-lnet test complete, duration 2863 sec ========== 09:11:00 (1789132260) [ 3159.073397] LNet: 144390:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3159.079270] LNetError: 144390:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3159.096129] LNet: Removed LNI 192.168.201.5@tcp [ 3159.990596] Key type .llcrypt unregistered [ 3159.996488] Key type ._llcrypt unregistered [ 3176.102301] Key type ._llcrypt registered [ 3176.109954] Key type .llcrypt registered [ 3176.823500] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 3176.855038] alg: No test for adler32 (adler32-zlib) [ 3178.058901] Lustre: Lustre: Build Version: 2.17.57_133_g066c38c [ 3178.392600] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 3180.120186] Key type lgssc registered [ 3181.566209] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3264.668461] Lustre: Mounted lustre-client - version 2.17.57_133_g066c38c [ 3270.640962] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3282.565248] Lustre: Unmounted lustre-client [ 3316.679895] Key type lgssc unregistered [ 3317.042499] LNet: 147355:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3317.048194] LNetError: 147355:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3317.076772] LNet: Removed LNI 192.168.201.5@tcp [ 3317.907142] Key type .llcrypt unregistered [ 3317.908517] Key type ._llcrypt unregistered