[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 428627569 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54d0-0x000f54df] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895288K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002240] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004014] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008014] pid_max: default: 32768 minimum: 301 [ 0.009141] LSM: Security Framework initializing [ 0.010061] Yama: becoming mindful. [ 0.011044] SELinux: Initializing. [ 0.013055] *** VALIDATE selinux *** [ 0.021872] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026092] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028189] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029132] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031098] *** VALIDATE tmpfs *** [ 0.032238] *** VALIDATE proc *** [ 0.033225] *** VALIDATE cgroup *** [ 0.033951] *** VALIDATE cgroup2 *** [ 0.034258] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035134] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037024] Spectre V2 : User space: Vulnerable [ 0.039003] Speculative Store Bypass: Vulnerable [ 0.041349] debug: unmapping init [mem 0xffffffff97059000-0xffffffff97060fff] [ 0.043833] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044531] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045022] ... version: 2 [ 0.045868] ... bit width: 48 [ 0.046014] ... generic registers: 4 [ 0.046797] ... value mask: 0000ffffffffffff [ 0.047021] ... max period: 00007fffffffffff [ 0.048012] ... fixed-purpose events: 3 [ 0.048706] ... event mask: 000000070000000f [ 0.049303] rcu: Hierarchical SRCU implementation. [ 0.051478] smp: Bringing up secondary CPUs ... [ 0.052682] x86: Booting SMP configuration: [ 0.053048] .... node #0, CPUs: #1 #2 #3 [ 0.056443] smp: Brought up 1 node, 4 CPUs [ 0.058016] smpboot: Max logical packages: 1 [ 0.059026] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.126937] node 0 deferred pages initialised in 66ms [ 0.130017] devtmpfs: initialized [ 0.131310] x86/mm: Memory block size: 128MB [ 0.134648] gcov: version magic: 0x41383552 [ 0.137116] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.140089] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.142272] pinctrl core: initialized pinctrl subsystem [ 0.144209] [ 0.144735] ************************************************************* [ 0.146012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.147009] ** ** [ 0.149010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.150008] ** ** [ 0.152015] ** This means that this kernel is built to expose internal ** [ 0.154012] ** IOMMU data structures, which may compromise security on ** [ 0.155009] ** your system. ** [ 0.157012] ** ** [ 0.158009] ** If you see this message and you are not debugging the ** [ 0.160010] ** kernel, report this immediately to your vendor! ** [ 0.161008] ** ** [ 0.163010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.164008] ************************************************************* [ 0.166687] NET: Registered protocol family 16 [ 0.167400] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.170046] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.171040] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.174094] cpuidle: using governor menu [ 0.175479] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.177308] PCI: Using configuration type 1 for base access [ 0.179146] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.187104] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.189035] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.192088] cryptd: max_cpu_qlen set to 1000 [ 0.194162] ACPI: Added _OSI(Module Device) [ 0.196018] ACPI: Added _OSI(Processor Device) [ 0.197080] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.199012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.203559] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.207352] ACPI: Interpreter enabled [ 0.208053] ACPI: PM: (supports S0 S3 S4 S5) [ 0.209011] ACPI: Using IOAPIC for interrupt routing [ 0.211096] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.214390] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.225480] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.228069] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.230017] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.233079] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.238492] acpiphp: Slot [2] registered [ 0.240100] acpiphp: Slot [5] registered [ 0.241083] acpiphp: Slot [6] registered [ 0.242098] acpiphp: Slot [3] registered [ 0.243032] acpiphp: Slot [4] registered [ 0.243985] acpiphp: Slot [7] registered [ 0.244075] acpiphp: Slot [8] registered [ 0.244828] acpiphp: Slot [9] registered [ 0.246082] acpiphp: Slot [10] registered [ 0.247060] acpiphp: Slot [11] registered [ 0.247751] acpiphp: Slot [12] registered [ 0.249146] acpiphp: Slot [13] registered [ 0.250084] acpiphp: Slot [14] registered [ 0.251087] acpiphp: Slot [15] registered [ 0.252089] acpiphp: Slot [16] registered [ 0.253080] acpiphp: Slot [17] registered [ 0.254148] acpiphp: Slot [18] registered [ 0.255110] acpiphp: Slot [19] registered [ 0.256124] acpiphp: Slot [20] registered [ 0.257061] acpiphp: Slot [21] registered [ 0.257936] acpiphp: Slot [22] registered [ 0.259055] acpiphp: Slot [23] registered [ 0.259764] acpiphp: Slot [24] registered [ 0.261089] acpiphp: Slot [25] registered [ 0.261998] acpiphp: Slot [26] registered [ 0.263060] acpiphp: Slot [27] registered [ 0.264064] acpiphp: Slot [28] registered [ 0.265087] acpiphp: Slot [29] registered [ 0.266136] acpiphp: Slot [30] registered [ 0.268130] acpiphp: Slot [31] registered [ 0.270124] PCI host bridge to bus 0000:00 [ 0.272035] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.274028] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.276018] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.278022] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.280024] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.282064] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.283144] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.284818] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.286939] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.292808] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.295369] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.297016] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.299017] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.302019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.303535] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.306751] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.308037] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.311370] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.314014] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.322016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.327012] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.331295] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.337019] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.342016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.355017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.362063] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.367026] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.371019] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.382018] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.394159] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.397405] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.399452] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.401354] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.405272] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.412028] iommu: Default domain type: Passthrough [ 0.414299] SCSI subsystem initialized [ 0.415151] ACPI: bus type USB registered [ 0.417132] usbcore: registered new interface driver usbfs [ 0.419145] usbcore: registered new interface driver hub [ 0.421097] usbcore: registered new device driver usb [ 0.423201] pps_core: LinuxPPS API ver. 1 registered [ 0.425013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.429063] PTP clock support registered [ 0.432052] EDAC MC: Ver: 3.0.0 [ 0.434146] PCI: Using ACPI for IRQ routing [ 0.435864] NetLabel: Initializing [ 0.438012] NetLabel: domain hash size = 128 [ 0.439014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.441083] NetLabel: unlabeled traffic allowed by default [ 0.443191] vgaarb: loaded [ 0.445363] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.447013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.456342] clocksource: Switched to clocksource kvm-clock [ 0.580763] VFS: Disk quotas dquot_6.6.0 [ 0.581764] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.583253] *** VALIDATE ramfs *** [ 0.584413] *** VALIDATE hugetlbfs *** [ 0.585700] pnp: PnP ACPI init [ 0.587193] pnp: PnP ACPI: found 6 devices [ 0.613787] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.616211] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.617396] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.618597] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.619844] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.621062] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.623048] NET: Registered protocol family 2 [ 0.624816] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.628052] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.630181] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.633897] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.636310] TCP: Hash tables configured (established 65536 bind 65536) [ 0.638238] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.639979] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.641621] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.643357] NET: Registered protocol family 1 [ 0.644867] RPC: Registered named UNIX socket transport module. [ 0.645983] RPC: Registered udp transport module. [ 0.646890] RPC: Registered tcp transport module. [ 0.647719] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.648918] NET: Registered protocol family 44 [ 0.650341] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.651779] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.652883] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.654384] PCI: CLS 0 bytes, default 64 [ 0.655607] Unpacking initramfs... [ 2.000636] debug: unmapping init [mem 0xffff8d8cfcc64000-0xffff8d8cfffcffff] [ 2.004452] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.006026] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.008327] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.487123] Initialise system trusted keyrings [ 2.488677] Key type blacklist registered [ 2.490833] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.499429] zbud: loaded [ 2.501940] *** VALIDATE nfs *** [ 2.503057] *** VALIDATE nfs4 *** [ 2.504340] pstore: using deflate compression [ 2.507911] Platform Keyring initialized [ 2.602548] NET: Registered protocol family 38 [ 2.603700] Key type asymmetric registered [ 2.604567] Asymmetric key parser 'x509' registered [ 2.605762] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.607849] io scheduler mq-deadline registered [ 2.609113] io scheduler kyber registered [ 2.610243] io scheduler bfq registered [ 2.611284] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.612954] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.614654] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.616338] ACPI: Power Button [PWRF] [ 2.619659] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.623543] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.628756] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.653105] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.680593] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.685369] Non-volatile memory driver v1.3 [ 2.687136] Linux agpgart interface v0.103 [ 2.716113] virtio_blk virtio1: [vda] 149768 512-byte logical blocks (76.7 MB/73.1 MiB) [ 2.718858] vda: detected capacity change from 0 to 76681216 [ 2.734200] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.737348] vdb: detected capacity change from 0 to 1073741824 [ 2.744200] libphy: Fixed MDIO Bus: probed [ 2.763953] usbcore: registered new interface driver usbserial_generic [ 2.765519] usbserial: USB Serial support registered for generic [ 2.767147] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.769908] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.771084] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.772686] mousedev: PS/2 mouse device common for all mice [ 2.777323] rtc_cmos 00:05: RTC can wake from S4 [ 2.779069] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.781970] rtc_cmos 00:05: registered as rtc0 [ 2.784384] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.786071] intel_pstate: CPU model not supported [ 2.787872] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.791441] hid: raw HID events driver (C) Jiri Kosina [ 2.793094] usbcore: registered new interface driver usbhid [ 2.794156] usbhid: USB HID core driver [ 2.794292] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.795088] drop_monitor: Initializing network drop monitor service [ 2.798902] Initializing XFRM netlink socket [ 2.799990] NET: Registered protocol family 10 [ 2.801874] Segment Routing with IPv6 [ 2.802955] NET: Registered protocol family 17 [ 2.804410] mpls_gso: MPLS GSO support [ 2.807916] RAS: Correctable Errors collector initialized. [ 2.810028] AVX version of gcm_enc/dec engaged. [ 2.811456] AES CTR mode by8 optimization enabled [ 2.882103] sched_clock: Marking stable (2882036526, 0)->(3726968515, -844931989) [ 2.885804] registered taskstats version 1 [ 2.887596] Loading compiled-in X.509 certificates [ 2.889284] zswap: loaded using pool lzo/zbud [ 2.914170] Key type big_key registered [ 2.925740] Key type encrypted registered [ 2.927483] ima: No TPM chip found, activating TPM-bypass! [ 2.928688] ima: Allocated hash algorithm: sha1 [ 2.929695] ima: No architecture policies found [ 2.930914] evm: Initialising EVM extended attributes: [ 2.932809] evm: security.selinux [ 2.934114] evm: security.ima [ 2.935291] evm: security.capability [ 2.936609] evm: HMAC attrs: 0x1 [ 2.938560] rtc_cmos 00:05: setting system clock to 2026-09-05 23:38:59 UTC (1788651539) [ 2.944684] debug: unmapping init [mem 0xffffffff98003000-0xffffffff981fffff] [ 2.947749] debug: unmapping init [mem 0xffffffff96d82000-0xffffffff97058fff] [ 2.954128] Write protecting the kernel read-only data: 28672k [ 2.957660] debug: unmapping init [mem 0xffffffff95403000-0xffffffff955fffff] [ 2.960224] debug: unmapping init [mem 0xffffffff95d14000-0xffffffff95dfffff] [ 2.990530] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 2.997975] systemd[1]: Detected virtualization kvm. [ 2.999993] systemd[1]: Detected architecture x86-64. [ 3.001905] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.025952] systemd[1]: No hostname configured. [ 3.027592] systemd[1]: Set hostname to . [ 3.029050] random: systemd: uninitialized urandom read (16 bytes read) [ 3.031415] systemd[1]: Initializing machine ID from random generator. [ 3.073847] random: ln: uninitialized urandom read (6 bytes read) [ 3.176653] random: systemd: uninitialized urandom read (16 bytes read) [ 3.179825] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.187955] systemd[1]: Started Memstrack Anylazing Service. [ OK ] Started Memstrack Anylazing Service. [ 3.199922] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Slices. [ OK ] Reached target Swap. Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. [ OK ] Reached target Initrd Root Device. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. [ OK ] Reached target Paths. Starting Journal Service... Starting Create Volatile Files and Directories... Starting Setup Virtual Console... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.765406] device-mapper: uevent: version 1.0.3 [ 3.767074] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.421468] virtio_net virtio0 ens2: renamed from eth0 [ 4.447253] scsi host0: ata_piix [ 4.470330] scsi host1: ata_piix [ 4.472027] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 4.474690] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 8.812644] [ 8.813554] random: fast init done dracut-initqueue[590]: RTNETLINK answers: File exists [ 9.441954] random: crng init done [ 9.443334] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 9.755191] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 10.773203] printk: systemd: 25 output lines suppressed due to ratelimiting [ 11.010939] SELinux: Disabled at runtime. [ 11.069532] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.078028] systemd[1]: Detected virtualization kvm. [ 11.079677] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.579504] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.582614] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.588519] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.594310] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.596937] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.603142] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.606541] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting POSIX Message Queue File System... [ OK ] Created slice system-getty.slice. Mounting Kernel Debug File System... [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice system-serial\x2dgetty.slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Mounting Huge Pages File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] [ 11.669193] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on RPCbind Server Activation Socket. Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.001429] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.249637] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.271987] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.368948] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.403579] EDAC sbridge: Ver: 1.1.2 [ 13.331341] Key type dns_resolver registered [ 13.630089] NFS: Registering the id_resolver key type [ 13.631978] Key type id_resolver registered [ 13.633536] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Network Manager... Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. [ 21.012068] hrtimer: interrupt took 3214512 ns Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg117-client login: [ 69.012811] libcfs: loading out-of-tree module taints kernel. [ 69.328909] Key type ._llcrypt registered [ 69.338301] Key type .llcrypt registered [ 69.815984] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 69.833817] alg: No test for adler32 (adler32-zlib) [ 71.288954] Lustre: Lustre: Build Version: 2.17.58_2_g557ce73 [ 72.152556] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 73.951363] Key type lgssc registered [ 75.525739] Lustre: Echo OBD driver; http://www.lustre.org/ [ 218.585706] Lustre: Mounted lustre-client - version 2.17.58_2_g557ce73 [ 223.974441] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 240.788059] Lustre: DEBUG MARKER: oleg117-client.virtnet: executing check_logdir /tmp/testlogs/ [ 244.191183] Lustre: lustre-OST0000-osc-ffff8d8d42a09000: disconnect after 23s idle [ 245.746398] Lustre: DEBUG MARKER: oleg117-client.virtnet: executing yml_node [ 250.125235] Lustre: DEBUG MARKER: Client: 2.17.58.2 [ 252.979159] Lustre: DEBUG MARKER: MDS: 2.17.58.2 [ 255.555550] Lustre: DEBUG MARKER: OSS: 2.17.58.2 [ 257.457287] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Sat Sep 5 19:43:12 EDT 2026 [ 273.778868] Lustre: Unmounted lustre-client [ 300.698687] Key type lgssc unregistered [ 300.967253] LNet: 8429:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 300.976112] LNetError: 8429:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 301.017448] LNet: Removed LNI 192.168.201.17@tcp [ 301.861170] Key type .llcrypt unregistered [ 301.863288] Key type ._llcrypt unregistered [ 330.715273] Key type ._llcrypt registered [ 330.717599] Key type .llcrypt registered [ 330.862673] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 330.872173] alg: No test for adler32 (adler32-zlib) [ 331.808689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 331.897267] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 331.912993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show [ 336.041299] Lustre: DEBUG MARKER: excepting tests: 253 254 410 [ 336.234232] LNet: 9311:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 336.246803] LNetError: 9311:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 336.282805] LNet: Removed LNI 192.168.201.17@tcp [ 336.960946] Key type .llcrypt unregistered [ 336.965462] Key type ._llcrypt unregistered [ 343.740374] Key type ._llcrypt registered [ 343.742644] Key type .llcrypt registered [ 344.919448] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 19:44:40 (1788651880) [ 345.233697] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 345.250575] alg: No test for adler32 (adler32-zlib) [ 346.141680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 346.207805] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 346.222663] LNet: 9720:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 346.246443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 346.294623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 346.311409] LNet: 9727:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 352.370676] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 19:44:47 (1788651887) [ 352.550610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 352.620950] LNet: 10305:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 352.647881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 352.714983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp [ 358.868474] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 19:44:54 (1788651894) [ 359.293964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 359.327618] LNet: 10890:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 359.358954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 359.438887] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 2.2.2.2@o2ib [ 366.136866] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 19:45:01 (1788651901) [ 366.230050] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 366.266599] LNet: 11469:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 366.297041] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 366.372859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib [ 372.793375] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 19:45:08 (1788651908) [ 373.149534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 373.169723] LNet: 12060:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 373.186663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 373.261607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp [ 373.723912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp [ 382.070405] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 19:45:16 (1788651916) [ 382.289913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 382.367517] LNet: 12648:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 382.457528] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 382.655452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 383.568471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 384.195789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib [ 391.459614] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 19:45:26 (1788651926) [ 391.919335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 391.967207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 392.072465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi,[6-12/4]@bxi3f [ 400.062528] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 19:45:35 (1788651935) [ 400.211862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 400.233416] LNet: 13830:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 400.236666] LNet: 13830:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 400.258426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 400.355993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp [ 400.392989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 400.861449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib [ 400.928126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 401.416798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp [ 401.485379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 402.132768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib [ 402.162278] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 402.419356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib [ 402.457938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 402.769331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni [ 402.818203] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 403.256561] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni [ 403.284786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 403.651958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi [ 403.690361] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 404.090688] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi [ 404.145796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 404.765834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f [ 404.791976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 405.070506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f --nid [8-12]@bxi3f [ 405.112829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 405.554246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi,[3-7]@bxi3f [ 405.599763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 405.858802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 405.885667] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 412.885540] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 19:45:47 (1788651947) [ 413.135251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 413.185792] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 413.320979] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp [ 413.352709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp [ 422.475770] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 19:45:57 (1788651957) [ 422.693575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 422.742233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 422.817700] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 422.861627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 430.468047] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 19:46:05 (1788651965) [ 430.608266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 430.665686] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 430.753734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp [ 430.772157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp [ 437.839513] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 19:46:12 (1788651972) [ 438.051899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 438.080331] LNet: 16276:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 438.093104] LNet: 16276:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 438.141756] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 438.268939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 438.325884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 445.983354] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 19:46:21 (1788651981) [ 446.645337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 446.726982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 446.822965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp [ 446.854338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp [ 453.914087] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 19:46:28 (1788651988) [ 454.258152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 454.346264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 454.435754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 454.498022] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 464.077741] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 19:46:38 (1788651998) [ 464.226468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 464.270583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 464.417628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 465.282860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 472.364331] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 19:46:47 (1788652007) [ 472.907855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 472.970239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 473.109405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 473.275723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 480.480077] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 19:46:55 (1788652015) [ 480.855236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 480.929845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 481.210760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 481.465682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 489.919841] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 19:47:04 (1788652024) [ 490.166208] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 490.218953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 490.585067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 490.939288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 503.813535] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 19:47:18 (1788652038) [ 504.010134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 504.066434] LNet: 20517:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 504.073309] LNet: 20517:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 6 previous similar messages [ 504.134860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 504.342180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 504.442864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 511.864495] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 19:47:27 (1788652047) [ 512.079226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 512.143918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 512.250061] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 512.337001] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 519.041683] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 19:47:34 (1788652054) [ 519.247456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 519.321884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 519.435832] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 19@gni [ 526.209844] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 19:47:41 (1788652061) [ 526.537747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 526.579782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 526.687905] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib [ 534.048788] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 19:47:48 (1788652068) [ 534.440640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 534.507623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 534.603932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni [ 535.123817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni [ 542.640714] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 19:47:57 (1788652077) [ 542.804898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 542.884640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 542.992452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni [ 543.021283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 22@gni --nid 26@gni [ 550.589060] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 19:48:05 (1788652085) [ 550.912563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 551.015480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 551.102927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni [ 551.132192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni [ 557.818179] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 19:48:13 (1788652093) [ 558.079983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 558.132714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 558.236399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni [ 558.277907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp [ 566.229736] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 19:48:21 (1788652101) [ 566.676209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 566.732845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 566.856674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 566.905976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 574.421509] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 19:48:29 (1788652109) [ 574.923817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 574.956788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 575.053333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim [ 575.118698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp [ 575.137220] LNetError: 25839:0:(peer.c:2366:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 575.185662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --force [ 575.223748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show --nid 1.1.1.1@tcp [ 581.417536] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 19:48:36 (1788652116) [ 581.560232] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 581.577653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 581.636994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.G37XYKGimS [ 581.698348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.G37XYKGimS [ 581.745941] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.G37XYKGimS [ 588.278569] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 19:48:43 (1788652123) [ 588.419682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 588.471748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 588.532399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp [ 588.559232] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp [ 588.594971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer list [ 595.447112] Lustre: DEBUG MARKER: == sanity-lnet test 40: LNetAddPeer merges a pre-existing non-primary peer NID ========================================================== 19:48:50 (1788652130) [ 595.572471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 595.649163] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 595.695897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 595.720616] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 597.331593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 598.668182] Lustre: Lustre: Build Version: 2.17.58_2_g557ce73 [ 600.535743] Key type lgssc registered [ 601.841792] Lustre: Echo OBD driver; http://www.lustre.org/ [ 605.720075] LNetError: 27600:0:(lib-move.c:2151:lnet_handle_find_routed_path()) peer 192.168.201.17@tcp99 has no available nets [ 608.430685] Key type lgssc unregistered [ 608.740572] LNetError: 28706:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 608.761930] LNet: Removed LNI 192.168.201.17@tcp [ 609.324467] Key type .llcrypt unregistered [ 609.326818] Key type ._llcrypt unregistered [ 615.778938] Key type ._llcrypt registered [ 615.781857] Key type .llcrypt registered [ 617.251248] Lustre: DEBUG MARKER: == sanity-lnet test 50: Enable/disable routing via yaml import ========================================================== 19:49:12 (1788652152) [ 617.794848] Key type .llcrypt unregistered [ 617.798506] Key type ._llcrypt unregistered [ 624.216287] Key type ._llcrypt registered [ 624.217885] Key type .llcrypt registered [ 624.567620] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 624.595053] alg: No test for adler32 (adler32-zlib) [ 625.530443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 625.587661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 625.623463] LNet: 29617:0:(router.c:1481:lnet_nrb_tiny_calculate()) tiny_router_buffers=16 less than recommended minimum 512 [ 625.636290] LNet: 29617:0:(router.c:1503:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096 [ 625.642606] LNet: 29617:0:(router.c:1525:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256 [ 625.652904] LNet: Message forwarding will be enabled in 2s [ 628.767177] LNet: Message forwarding enabled [ 629.102401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 629.139079] LNet: Message forwarding will stop in 2s to 6s [ 632.865695] LNet: Message forwarding disabled [ 633.770826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 633.793845] LNet: 29715:0:(router.c:1481:lnet_nrb_tiny_calculate()) tiny_router_buffers=16 less than recommended minimum 512 [ 633.801819] LNet: 29715:0:(router.c:1503:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096 [ 633.810448] LNet: 29715:0:(router.c:1525:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256 [ 633.821908] LNet: Message forwarding will be enabled in 2s [ 634.031047] LNet: 29728:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 634.735835] Key type .llcrypt unregistered [ 634.738366] Key type ._llcrypt unregistered [ 648.065166] Key type ._llcrypt registered [ 648.072388] Key type .llcrypt registered [ 649.414907] Lustre: DEBUG MARKER: == sanity-lnet test 51: Set numa range via yaml import === 19:49:44 (1788652184) [ 650.249890] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 650.262820] alg: No test for adler32 (adler32-zlib) [ 651.215393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 651.344521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-51.yaml [ 657.821275] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 19:49:52 (1788652192) [ 657.966677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 658.010142] LNet: 31124:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 658.044383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 658.181446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid foobar [ 658.234459] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid foobar [ 658.276263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@o2ib [ 658.341599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --nid 1.1.1.1@tcp [ 658.372145] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --nid 1.1.1.1@tcp [ 658.441626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add [ 658.488953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 658.541624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp [ 658.595574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@foo [ 658.655282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@tcp [ 658.692226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@o2ib [ 658.746542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@gni [ 658.809199] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni [ 658.836302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni [ 658.889580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp [ 658.922068] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp [ 658.965551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp [ 659.007391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 659.039242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp --gateway 1@gni [ 659.075757] LNetError: 31172:0:(router.c:625:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni [ 665.318372] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 19:50:00 (1788652200) [ 665.697144] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 665.735563] LNet: 31743:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 665.770562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 665.907692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 673.302079] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 19:50:08 (1788652208) [ 675.448493] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_100 Need at least 2 remote nodes found oleg117-server [ 677.794137] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 19:50:12 (1788652212) [ 677.899355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 677.941868] LNet: 32667:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 677.976583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 678.089468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 678.135130] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 679.050888] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp1 --gateway 192.168.201.[18-22]@tcp [ 679.075336] LNet: 32808:0:(router.c:729:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 679.133685] LNetError: 32810:0:(lib-lnet.h:1281:lnet_set_route_aliveness()) route to tcp1 through 192.168.201.21@tcp has gone from up to down [ 679.160915] LNetError: 32677:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.19@tcp is being used as a gateway but routing feature is not turned on [ 687.172836] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 19:50:22 (1788652222) [ 689.481719] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_102 Need at least 2 remote nodes found oleg117-server [ 691.266298] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 19:50:26 (1788652226) [ 691.405804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 691.444304] LNet: 33740:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 691.451977] LNetError: 33740:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 691.493590] LNet: Removed LNI 192.168.201.17@tcp [ 691.570559] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 691.655591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 691.677542] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 692.329855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp103 --gateway 192.168.201.[18-23/2]@tcp [ 692.368555] LNet: 33863:0:(router.c:729:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 692.375782] LNet: 33863:0:(router.c:729:lnet_add_route()) Skipped 4 previous similar messages [ 692.411658] LNetError: 33750:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.22@tcp is being used as a gateway but routing feature is not turned on [ 692.423551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route del --net tcp103 --gateway 192.168.201.[18-23/2]@tcp [ 692.429191] LNetError: 33750:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 7 previous similar messages [ 692.429218] LNetError: 33750:0:(lib-lnet.h:1281:lnet_set_route_aliveness()) route to tcp103 through 192.168.201.22@tcp has gone from up to down [ 692.429223] LNetError: 33750:0:(lib-lnet.h:1281:lnet_set_route_aliveness()) Skipped 4 previous similar messages [ 698.830610] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 19:50:34 (1788652234) [ 698.983449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 699.003384] LNet: 34455:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 699.013833] LNetError: 34455:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 699.027888] LNet: Removed LNI 192.168.201.17@tcp [ 699.050396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 699.151370] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking -1 [ 699.233423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 699.271894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 699.318326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 699.356324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 699.381523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 699.422153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 0 [ 699.560456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 699.612964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 699.691533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 699.800341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 699.836978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 699.882682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 1 [ 699.983695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 700.024845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 700.083284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 700.171835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 700.211765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 700.302881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 2 [ 700.418576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 700.461857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 700.557309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 700.617532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 700.664530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 700.696143] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 700.737359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 700.762336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 700.811745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 700.897478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 700.927507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 700.976784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 4 [ 701.018479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 701.040759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 701.087896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 707.238523] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 19:50:42 (1788652242) [ 709.374604] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_105 Need at least 2 remote nodes found oleg117-server [ 714.029516] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 19:50:47 (1788652247) [ 717.129335] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_106 Need at least 2 remote nodes found oleg117-server [ 718.897283] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 19:50:54 (1788652254) [ 719.224523] LNet: 35857:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 719.228917] LNet: 35857:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 11 previous similar messages [ 719.940219] Key type .llcrypt unregistered [ 719.942416] Key type ._llcrypt unregistered [ 729.196040] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 729.430529] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 729.440522] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 729.448487] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 732.333936] Key type ._llcrypt registered [ 732.338212] Key type .llcrypt registered [ 732.621827] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 732.660763] alg: No test for adler32 (adler32-zlib) [ 733.975905] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 734.171189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 734.231270] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 734.920267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 734.930239] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 736.879800] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 736.952621] LNet: Removed LNI 10.1.2.3@tcp [ 737.143823] LNet: 36575:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 737.148702] LNetError: 36575:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 737.184783] LNet: Removed LNI 192.168.201.17@tcp [ 737.913948] Key type .llcrypt unregistered [ 737.915435] Key type ._llcrypt unregistered [ 750.242910] Key type ._llcrypt registered [ 750.245333] Key type .llcrypt registered [ 751.835364] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 19:51:26 (1788652286) [ 752.591334] Key type .llcrypt unregistered [ 752.593993] Key type ._llcrypt unregistered [ 759.615541] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 759.781834] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 759.789810] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 759.807582] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 762.631707] Key type ._llcrypt registered [ 762.634766] Key type .llcrypt registered [ 762.902942] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 762.925771] alg: No test for adler32 (adler32-zlib) [ 763.844993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 763.895780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 763.921797] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 764.573881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 764.592678] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 767.173920] LNet: 38067:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 767.186981] LNetError: 38067:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 767.196788] LNet: Removed LNI 10.1.2.3@tcp [ 767.211177] LNet: Removed LNI 192.168.201.17@tcp [ 767.878573] Key type .llcrypt unregistered [ 767.880420] Key type ._llcrypt unregistered [ 781.087704] Key type ._llcrypt registered [ 781.089721] Key type .llcrypt registered [ 782.696647] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 19:51:57 (1788652317) [ 783.188315] Key type .llcrypt unregistered [ 783.190377] Key type ._llcrypt unregistered [ 789.235506] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 789.546101] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 789.604500] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 789.626553] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 792.333514] Key type ._llcrypt registered [ 792.335885] Key type .llcrypt registered [ 792.628919] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 792.640566] alg: No test for adler32 (adler32-zlib) [ 793.563879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 793.644682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 793.686357] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 795.283687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 795.301794] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 797.372832] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 797.407983] LNet: Removed LNI 10.1.2.3@tcp [ 797.436899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg:0 [ 797.469327] LNet: Removed LNI 10.1.2.31@tcp [ 797.498772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 797.540562] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 798.314651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 798.335834] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 799.504257] LNet: 39983:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 799.527392] LNetError: 39983:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg:0 not found [ 799.537476] LNet: Removed LNI 10.1.2.31@tcp [ 799.548314] LNet: Removed LNI 10.1.2.3@tcp [ 800.391880] Key type .llcrypt unregistered [ 800.399930] Key type ._llcrypt unregistered [ 816.299596] Key type ._llcrypt registered [ 816.301963] Key type .llcrypt registered [ 817.814743] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 19:52:32 (1788652352) [ 818.140107] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 818.165505] alg: No test for adler32 (adler32-zlib) [ 819.208879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 819.264697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 819.282825] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 820.151965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 820.169860] LNet: 40934:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 820.174866] LNetError: 40934:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 820.189452] LNet: Removed LNI 192.168.201.17@tcp [ 820.223237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 820.332607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --nid 192.168.201.17@tcp [ 820.382696] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 821.246051] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --nid 192.168.201.17@tcp [ 821.275697] LNet: Removed LNI 192.168.201.17@tcp [ 821.342440] LNet: 41069:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 822.271553] Key type .llcrypt unregistered [ 822.273573] Key type ._llcrypt unregistered [ 832.888351] Key type ._llcrypt registered [ 832.891292] Key type .llcrypt registered [ 834.202938] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 19:52:49 (1788652369) [ 836.054398] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_111 Need at least 2 remote nodes found oleg117-server [ 838.253186] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 19:52:53 (1788652373) [ 838.804330] Key type .llcrypt unregistered [ 838.806722] Key type ._llcrypt unregistered [ 846.736081] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 846.890699] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 846.893862] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 846.897787] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 849.622189] Key type ._llcrypt registered [ 849.627704] Key type .llcrypt registered [ 849.910665] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 849.926660] alg: No test for adler32 (adler32-zlib) [ 850.877638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 850.959453] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 851.714714] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180] [ 854.015757] LNet: 42901:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 854.036658] LNetError: 42901:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 854.041652] LNet: Removed LNI 192.168.201.17@tcp1 [ 854.067699] LNet: Removed LNI 10.1.2.3@tcp2 [ 855.003586] Key type .llcrypt unregistered [ 855.005987] Key type ._llcrypt unregistered [ 868.570519] Key type ._llcrypt registered [ 868.575508] Key type .llcrypt registered [ 869.976544] Lustre: DEBUG MARKER: == sanity-lnet test 150: Check import of ip2nets yaml sequence ========================================================== 19:53:25 (1788652405) [ 870.388471] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 870.403431] alg: No test for adler32 (adler32-zlib) [ 871.313880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 871.365370] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 871.382952] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 872.158444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 872.181410] LNet: 43885:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 872.185703] LNetError: 43885:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 872.204521] LNet: Removed LNI 192.168.201.17@tcp [ 872.217665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 872.278231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet.yaml [ 872.294952] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 879.399686] Lustre: DEBUG MARKER: == sanity-lnet test 151: Check correct application of multiple ip2nets rules ========================================================== 19:53:34 (1788652414) [ 879.636963] LNet: 44476:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 879.647982] LNetError: 44476:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 879.664445] LNet: Removed LNI 192.168.201.17@tcp [ 880.397802] Key type .llcrypt unregistered [ 880.400361] Key type ._llcrypt unregistered [ 886.504529] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 886.776604] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 886.784786] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 886.793735] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 889.362404] Key type ._llcrypt registered [ 889.367846] Key type .llcrypt registered [ 889.654163] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 889.665917] alg: No test for adler32 (adler32-zlib) [ 890.587611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 890.682869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 890.719489] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 891.476984] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 891.501990] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 893.609387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 893.617264] LNet: 45217:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 893.620143] LNetError: 45217:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 893.622594] LNet: Removed LNI 10.1.2.3@tcp [ 893.632468] LNet: Removed LNI 192.168.201.17@tcp [ 893.642750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 893.703659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 893.727593] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 893.730985] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 894.098980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 894.110622] LNet: Removed LNI 10.1.2.3@tcp [ 894.120511] LNet: Removed LNI 192.168.201.17@tcp [ 894.142174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 894.191837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 894.214735] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 894.219884] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 894.485552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 894.502991] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 895.726489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 895.746313] LNet: 45406:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 895.754146] LNet: 45406:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 895.760102] LNetError: 45406:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 895.766172] LNetError: 45406:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 895.772526] LNet: Removed LNI 10.1.2.3@tcp [ 895.775426] LNet: Removed LNI 192.168.201.17@tcp [ 895.789512] LNet: Removed LNI 192.168.201.17@tcp2 [ 895.818471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 895.890465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 895.938595] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 895.951479] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 895.960717] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 904.854711] Lustre: DEBUG MARKER: == sanity-lnet test 152: Check import of ip2nets with wildcard IP patterns ========================================================== 19:53:59 (1788652439) [ 904.946761] LNet: 45998:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 904.959866] LNetError: 45998:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 904.965508] LNetError: 45998:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 905.002731] LNet: Removed LNI 10.1.2.3@tcp [ 905.005582] LNet: Removed LNI 192.168.201.17@tcp [ 905.034185] LNet: Removed LNI 192.168.201.17@tcp2 [ 905.844099] Key type .llcrypt unregistered [ 905.848890] Key type ._llcrypt unregistered [ 914.106033] Key type ._llcrypt registered [ 914.109826] Key type .llcrypt registered [ 914.707765] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 914.749883] alg: No test for adler32 (adler32-zlib) [ 915.920468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 916.058159] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 916.116274] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 917.691760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 917.727318] LNet: 46412:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 917.736756] LNetError: 46412:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 917.786137] LNet: Removed LNI 192.168.201.17@tcp [ 917.813552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 917.921631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 917.976833] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 919.298407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 919.332881] LNet: 46442:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 919.353646] LNetError: 46442:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 919.397220] LNet: Removed LNI 192.168.201.17@tcp [ 919.445982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 919.516448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 919.566685] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 920.186929] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 920.235084] LNet: Removed LNI 192.168.201.17@tcp [ 920.263309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 920.371416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 920.413740] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 921.096211] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 921.162317] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 921.163976] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 921.165959] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 923.652781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 923.688759] LNet: 46552:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 923.696767] LNet: 46552:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 923.708779] LNetError: 46552:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 923.718988] LNetError: 46552:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 923.736220] LNet: Removed LNI 192.168.201.17@tcp [ 923.763124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 923.850055] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 923.898369] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 932.916920] Lustre: DEBUG MARKER: == sanity-lnet test 153: Check import of ip2nets with IP address ranges ========================================================== 19:54:27 (1788652467) [ 933.071690] LNet: 47140:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 933.078675] LNetError: 47140:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 933.111727] LNet: Removed LNI 192.168.201.17@tcp [ 933.994154] Key type .llcrypt unregistered [ 933.997770] Key type ._llcrypt unregistered [ 940.578792] Key type ._llcrypt registered [ 940.580751] Key type .llcrypt registered [ 940.745363] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 940.757921] alg: No test for adler32 (adler32-zlib) [ 941.791417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 941.886812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 941.913566] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 942.692668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 942.707538] LNet: 47544:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 942.711733] LNetError: 47544:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 942.731393] LNet: Removed LNI 192.168.201.17@tcp [ 942.768657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 942.851276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-153.yaml [ 942.905891] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 949.425287] Lustre: DEBUG MARKER: == sanity-lnet test 154: Check import of ip2nets with stepped IP address ranges ========================================================== 19:54:44 (1788652484) [ 949.792652] LNet: 48136:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 949.808539] LNetError: 48136:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 949.837450] LNet: Removed LNI 192.168.201.17@tcp [ 950.639380] Key type .llcrypt unregistered [ 950.641521] Key type ._llcrypt unregistered [ 956.455293] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 956.533930] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 956.536354] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 956.546895] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 959.065460] Key type ._llcrypt registered [ 959.068157] Key type .llcrypt registered [ 959.226869] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 959.241372] alg: No test for adler32 (adler32-zlib) [ 960.185562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 960.274051] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 960.337817] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 961.126985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 961.135727] LNet: 48584:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 961.138832] LNetError: 48584:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 961.150502] LNet: Removed LNI 192.168.201.17@tcp [ 961.161542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 961.210566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-154.yaml [ 961.234598] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 971.138874] Lustre: DEBUG MARKER: == sanity-lnet test 155: Check ip2nets import failure with no matching rules ========================================================== 19:55:05 (1788652505) [ 971.269626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 971.290496] LNet: 49175:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 971.299287] LNetError: 49175:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 971.326416] LNet: Removed LNI 192.168.201.17@tcp [ 971.372374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 971.564766] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-155.yaml [ 978.211483] Lustre: DEBUG MARKER: == sanity-lnet test 156: Check import of ip2nets with comma-separated IP ranges ========================================================== 19:55:13 (1788652513) [ 978.509783] LNet: 49768:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 978.957580] Key type .llcrypt unregistered [ 978.959976] Key type ._llcrypt unregistered [ 985.160529] Key type ._llcrypt registered [ 985.164360] Key type .llcrypt registered [ 985.402542] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 985.424992] alg: No test for adler32 (adler32-zlib) [ 986.358346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 986.419702] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 986.458654] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 987.038698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 987.054817] LNet: 50162:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 987.064456] LNetError: 50162:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 987.082963] LNet: Removed LNI 192.168.201.17@tcp [ 987.099394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 987.165845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-156.yaml [ 987.210035] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 993.397492] Lustre: DEBUG MARKER: == sanity-lnet test 157: Check import failure with malformed ip2nets YAML ========================================================== 19:55:28 (1788652528) [ 993.667271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 993.688948] LNet: 50756:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 993.695324] LNetError: 50756:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 993.705924] LNet: Removed LNI 192.168.201.17@tcp [ 993.721534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 993.764774] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-157.yaml [ 999.135380] Lustre: DEBUG MARKER: == sanity-lnet test 158: Check import of ip2nets with single IPv6 address ========================================================== 19:55:34 (1788652534) [ 1000.417283] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_158 Interface has no IPv6 [ 1002.507992] Lustre: DEBUG MARKER: == sanity-lnet test 159: Check import of ip2nets with IPv6 /128 CIDR notation ========================================================== 19:55:37 (1788652537) [ 1003.838535] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_159 Interface has no IPv6 [ 1005.556340] Lustre: DEBUG MARKER: == sanity-lnet test 160: Check correct application of multiple IPv6 ip2nets rules ========================================================== 19:55:40 (1788652540) [ 1007.772786] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_160 Interface has no IPv6 [ 1009.661437] Lustre: DEBUG MARKER: == sanity-lnet test 161: Check import of ip2nets with various IPv6 prefix lengths ========================================================== 19:55:44 (1788652544) [ 1011.945967] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_161 Interface has no IPv6 [ 1013.748434] Lustre: DEBUG MARKER: == sanity-lnet test 162: Check ip2nets import failure with non-matching IPv6 ========================================================== 19:55:48 (1788652548) [ 1015.220582] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_162 Interface has no IPv6 [ 1017.273302] Lustre: DEBUG MARKER: == sanity-lnet test 170: Check CPTs and tunables parsing when importing ========================================================== 19:55:52 (1788652552) [ 1017.544801] LNet: 52809:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1017.973630] Key type .llcrypt unregistered [ 1017.974841] Key type ._llcrypt unregistered [ 1025.353935] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1025.586190] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1025.605154] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1025.620197] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1028.237442] Key type ._llcrypt registered [ 1028.246920] Key type .llcrypt registered [ 1028.507240] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1028.536969] alg: No test for adler32 (adler32-zlib) [ 1029.418763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1029.546835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1029.568554] LNet: Added LNI 192.168.201.17@tcp [8/512/0/180] [ 1029.584285] LNet: Added LNI 10.1.2.3@tcp [8/512/0/180] [ 1029.589607] LNet: 53128:0:(api-ni.c:2991:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1029.740571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1029.765623] LNet: 53147:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1029.772848] LNetError: 53147:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1029.781646] LNet: Removed LNI 10.1.2.3@tcp [ 1029.785142] LNetError: 53147:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1029.831654] LNet: Removed LNI 192.168.201.17@tcp [ 1029.845092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1029.916182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1029.948718] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1029.986294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1030.003241] LNet: 53171:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1030.010141] LNetError: 53171:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1030.021318] LNet: Removed LNI 192.168.201.17@tcp [ 1030.040880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1030.133847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1030.162969] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1030.167877] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1030.170326] LNet: 53179:0:(api-ni.c:2991:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1030.292937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1030.333963] LNet: 53197:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1030.346232] LNetError: 53197:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1030.359791] LNet: Removed LNI 10.1.2.3@tcp [ 1030.363383] LNetError: 53197:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1030.388224] LNet: Removed LNI 192.168.201.17@tcp [ 1030.431547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1030.567558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1030.615880] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1030.629456] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1030.636322] LNet: 53205:0:(api-ni.c:2991:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1030.897354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1030.923554] LNet: 53231:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1030.928499] LNetError: 53231:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1030.934116] LNet: Removed LNI 10.1.2.3@tcp [ 1030.940135] LNetError: 53231:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1030.958279] LNet: Removed LNI 192.168.201.17@tcp [ 1030.995961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1031.072611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1031.100459] LNet: Added LNI 192.168.201.17@tcp [8/256/0/182] [ 1031.104744] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 1031.234848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1031.267503] LNet: 53259:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1031.272563] LNetError: 53259:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1031.282880] LNet: Removed LNI 10.1.2.3@tcp [ 1031.284586] LNetError: 53259:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1031.317154] LNet: Removed LNI 192.168.201.17@tcp [ 1031.351746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1031.424072] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1031.444615] LNet: Added LNI 192.168.201.17@tcp [8/256/0/182] [ 1031.453241] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 1031.455192] LNet: 53267:0:(api-ni.c:2991:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1031.563318] LNet: Removed LNI 10.1.2.3@tcp [ 1031.583342] LNet: Removed LNI 192.168.201.17@tcp [ 1032.355798] Key type .llcrypt unregistered [ 1032.359190] Key type ._llcrypt unregistered [ 1046.008174] Key type ._llcrypt registered [ 1046.010313] Key type .llcrypt registered [ 1047.062865] Lustre: DEBUG MARKER: == sanity-lnet test 180: FastReg map error should not leak descriptors ========================================================== 19:56:22 (1788652582) [ 1048.948173] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_180 Need o2ib NETTYPE [ 1050.402542] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 19:56:25 (1788652585) [ 1050.894969] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1050.912150] alg: No test for adler32 (adler32-zlib) [ 1051.804251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1051.873336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1051.900044] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1052.727406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --ip2net tcp 192.168.201.17 [ 1052.747526] LNet: 54530:0:(config.c:1740:lnet_parse_ip2nets()) Kernel parsing of ip2nets is deprecated. Consider converting to LNet YAML configuration. [ 1052.754469] LNet: Removed LNI 192.168.201.17@tcp [ 1059.835138] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 19:56:34 (1788652594) [ 1059.929180] LNet: 55093:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1059.940492] LNetError: 55093:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1059.953868] LNet: Removed LNI 192.168.201.17@tcp [ 1060.551710] Key type .llcrypt unregistered [ 1060.553433] Key type ._llcrypt unregistered [ 1068.785240] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1068.954063] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1071.606603] Key type ._llcrypt registered [ 1071.612127] Key type .llcrypt registered [ 1071.824583] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1071.861606] alg: No test for adler32 (adler32-zlib) [ 1073.330083] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1080.600387] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 19:56:55 (1788652615) [ 1080.945533] LNet: 56010:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1080.974909] LNetError: 56010:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1081.009949] LNet: Removed LNI 10.1.2.3@tcp [ 1081.991421] Key type .llcrypt unregistered [ 1081.994174] Key type ._llcrypt unregistered [ 1088.085852] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1088.286725] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1088.290162] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1088.298108] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1090.847792] Key type ._llcrypt registered [ 1090.848719] Key type .llcrypt registered [ 1090.936609] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1090.949641] alg: No test for adler32 (adler32-zlib) [ 1091.905535] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1098.289725] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 19:57:13 (1788652633) [ 1099.592807] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_202 kernel ip2nets deprecated in 2.17.53 [ 1101.611773] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 19:57:16 (1788652636) [ 1101.766459] LNet: 57187:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1101.779470] LNetError: 57187:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1101.806092] LNet: Removed LNI 10.1.2.3@tcp [ 1102.884534] Key type .llcrypt unregistered [ 1102.886406] Key type ._llcrypt unregistered [ 1110.554690] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1110.827500] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1110.831613] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1110.842538] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1112.426804] Key type ._llcrypt registered [ 1112.430259] Key type .llcrypt registered [ 1112.675196] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1112.686991] alg: No test for adler32 (adler32-zlib) [ 1113.600961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1113.665309] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1122.200222] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 19:57:37 (1788652657) [ 1122.724380] LNet: 58411:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1122.744483] LNetError: 58411:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1122.781817] LNet: Removed LNI 10.1.2.3@tcp [ 1123.461554] Key type .llcrypt unregistered [ 1123.465641] Key type ._llcrypt unregistered [ 1129.058392] Key type ._llcrypt registered [ 1129.061238] Key type .llcrypt registered [ 1129.364594] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1129.392274] alg: No test for adler32 (adler32-zlib) [ 1130.358729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1130.395989] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1138.040550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1138.829306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_interrupt -r 1 [ 1138.897694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_interrupt -r 1 [ 1138.953607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1138.984782] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1139.270487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1139.318816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1139.363263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1139.395986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1139.439037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1139.474153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1139.521229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1139.552319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1139.600727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1139.616272] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1139.618724] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1139.623123] LNet: 58847:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1139.647189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1139.694643] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1139.739059] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1139.773395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1139.819295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1139.855356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1139.904947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1140.391268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_dropped -r 1 [ 1140.412967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_dropped -r 1 [ 1140.440366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1140.703334] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1140.739707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1140.769141] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1140.810922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1140.854593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1140.897752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1140.926180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1140.965403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1140.990284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1141.004573] LNet: 58958:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1141.027583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1141.081360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1141.112638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1141.136605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1141.188846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1141.228809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1141.262442] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1141.790915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_aborted -r 1 [ 1141.825486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_aborted -r 1 [ 1141.857115] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1141.876171] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1141.879449] LNet: Skipped 1 previous similar message [ 1142.143541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1142.185925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1142.224742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1142.265467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1142.294990] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1142.313423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1142.336935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1142.361462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1142.391565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1142.421694] LNet: 59070:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1142.459271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1142.500814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1142.532986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1142.581445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1142.645810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1142.696953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1142.738492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1143.246740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_no_route -r 1 [ 1143.298852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_no_route -r 1 [ 1143.360901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1143.614067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1143.636527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1143.662512] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1143.703109] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1143.746108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1143.770662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1143.788253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1143.818664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1143.847244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1143.880949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1143.904309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1143.933088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1143.966478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1143.999986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1144.028730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1144.059974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1144.615584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_timeout -r 1 [ 1144.643337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_timeout -r 1 [ 1144.678610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1144.697635] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1144.705958] LNet: Skipped 2 previous similar messages [ 1144.943310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1144.972066] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1145.014889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1145.064170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1145.123532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1145.158936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1145.196592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1145.272413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1145.321825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1145.343515] LNet: 59292:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1145.349661] LNet: 59292:0:(api-ni.c:301:recovery_interval_set()) Skipped 1 previous similar message [ 1145.364121] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1145.403284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1145.444428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1145.493419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1145.557860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1145.611190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1145.664640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1146.135803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_error -r 1 [ 1146.169327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_error -r 1 [ 1146.199721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1146.399866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1146.433779] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1146.456831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1146.481215] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1146.502193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1146.530245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1146.552206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1146.576620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1146.601623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1146.634428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1146.662870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1146.697583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1146.730793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1146.761412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1146.798509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1146.836972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1147.078248] LNet: 59445:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1147.090195] LNetError: 59445:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1148.132775] LNet: Removed LNI 192.168.201.17@tcp [ 1148.621546] Key type .llcrypt unregistered [ 1148.625657] Key type ._llcrypt unregistered [ 1167.358916] Key type ._llcrypt registered [ 1167.361771] Key type .llcrypt registered [ 1168.569736] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 19:58:23 (1788652703) [ 1169.270846] Key type .llcrypt unregistered [ 1169.274773] Key type ._llcrypt unregistered [ 1175.421199] Key type ._llcrypt registered [ 1175.422981] Key type .llcrypt registered [ 1175.592359] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1175.606915] alg: No test for adler32 (adler32-zlib) [ 1176.529622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1176.542562] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1183.673477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1193.021374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1193.064704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1193.099419] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1194.324600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_interrupt -r 1 [ 1194.366216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_interrupt -r 1 [ 1194.437917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_interrupt -r 1 [ 1194.499325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_interrupt -r 1 [ 1194.560196] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_interrupt -r 1 [ 1194.640089] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_interrupt -r 1 [ 1194.698386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_interrupt -r 1 [ 1194.747406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_interrupt -r 1 [ 1194.810287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1194.833345] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1194.845825] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1195.076534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1195.108497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1195.143576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1195.172071] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1195.201905] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1195.230912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1195.296465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1195.348980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1195.411711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1195.434600] LNet: 60912:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1195.456332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1195.489294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1195.517070] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1195.540699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1195.584475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1195.630313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1195.653639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1196.204613] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_dropped -r 1 [ 1196.238808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_dropped -r 1 [ 1196.277684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_dropped -r 1 [ 1196.315288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_dropped -r 1 [ 1196.362900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_dropped -r 1 [ 1196.399963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_dropped -r 1 [ 1196.431484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_dropped -r 1 [ 1196.496235] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_dropped -r 1 [ 1196.567495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1196.592772] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1196.596265] LNet: Skipped 2 previous similar messages [ 1196.827769] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1196.853709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1196.877342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1196.894088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1196.917429] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1196.941761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1196.964132] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1196.989348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1197.029167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1197.046592] LNet: 61035:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1197.072287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1197.099632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1197.133746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1197.194732] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1197.234699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1197.268471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1197.296289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1197.778386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_aborted -r 1 [ 1197.811336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_aborted -r 1 [ 1197.831292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_aborted -r 1 [ 1197.850113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_aborted -r 1 [ 1197.901379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_aborted -r 1 [ 1197.948502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_aborted -r 1 [ 1198.001271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_aborted -r 1 [ 1198.046259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_aborted -r 1 [ 1198.085415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1198.109616] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1198.116896] LNet: Skipped 2 previous similar messages [ 1198.352808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1198.374400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1198.403219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1198.424151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1198.450573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1198.471738] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1198.493627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1198.516784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1198.539504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1198.550885] LNet: 61159:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1198.576376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1198.603450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1198.633524] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1198.657213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1198.690943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1198.716936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1198.742389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1199.179507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_no_route -r 1 [ 1199.215833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_no_route -r 1 [ 1199.258655] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_no_route -r 1 [ 1199.295091] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_no_route -r 1 [ 1199.327265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_no_route -r 1 [ 1199.365834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_no_route -r 1 [ 1199.405438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_no_route -r 1 [ 1199.446266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_no_route -r 1 [ 1199.485239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1199.703433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1199.728703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1199.754925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1199.778895] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1199.810741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1199.846328] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1199.868748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1199.900488] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1199.943140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1199.989615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1200.011257] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1200.035193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1200.076996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1200.105509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1200.138696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1200.164752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1200.715699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_timeout -r 1 [ 1200.744242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_timeout -r 1 [ 1200.776707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_timeout -r 1 [ 1200.810902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_timeout -r 1 [ 1200.845741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_timeout -r 1 [ 1200.876904] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_timeout -r 1 [ 1200.920776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_timeout -r 1 [ 1200.957377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_timeout -r 1 [ 1200.984202] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1200.994327] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1200.997377] LNet: Skipped 5 previous similar messages [ 1201.157857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1201.185934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1201.207880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1201.226751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1201.244320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1201.271691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1201.302621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1201.332179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1201.360189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1201.389127] LNet: 61405:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1201.396153] LNet: 61405:0:(api-ni.c:301:recovery_interval_set()) Skipped 1 previous similar message [ 1201.415347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1201.442438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1201.483782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1201.509351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1201.535299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1201.560714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1201.586698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1201.970648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_error -r 1 [ 1202.008977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_error -r 1 [ 1202.038965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_error -r 1 [ 1202.061739] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_error -r 1 [ 1202.077682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_error -r 1 [ 1202.096275] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_error -r 1 [ 1202.121091] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_error -r 1 [ 1202.152562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_error -r 1 [ 1202.182095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1202.355883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1202.399715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1202.424391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1202.447529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1202.469282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1202.492701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1202.515554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1202.542787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1202.569978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1202.600780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1202.630973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1202.658723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1202.681630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1202.703887] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1202.727608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1202.749542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1207.541788] LNet: 61604:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1207.567975] LNetError: 61604:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1207.584532] LNet: Removed LNI 192.168.201.17@tcp [ 1207.619220] LNet: Removed LNI 192.168.201.17@tcp1 [ 1208.258753] Key type .llcrypt unregistered [ 1208.271161] Key type ._llcrypt unregistered [ 1225.311952] Key type ._llcrypt registered [ 1225.315265] Key type .llcrypt registered [ 1227.196721] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 19:59:21 (1788652761) [ 1228.377388] Key type .llcrypt unregistered [ 1228.383175] Key type ._llcrypt unregistered [ 1236.149612] Key type ._llcrypt registered [ 1236.162803] Key type .llcrypt registered [ 1236.382928] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1236.409781] alg: No test for adler32 (adler32-zlib) [ 1237.445857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1237.493360] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1246.304287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1246.958585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_dropped -r 1 [ 1247.021706] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_dropped -r 1 [ 1247.077535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1247.114530] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1247.312763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1247.328358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1247.344292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1247.363738] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1247.385425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1247.411670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1247.432954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1247.461968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1247.511912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1247.548547] LNet: 62852:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1247.567962] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1247.590468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1247.627558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1247.656391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1247.726157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1247.757435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1247.808757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1248.492652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_error -r 1 [ 1248.537467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_error -r 1 [ 1248.582413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1248.627186] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1248.831338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1248.879669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1248.913860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1248.943489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1248.993988] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1249.027381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1249.059660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1249.095386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1249.142405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1249.165573] LNet: 62964:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1249.181903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1249.221360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1249.252974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1249.288471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1249.329372] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1249.378791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1249.417319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1249.945818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_timeout -r 1 [ 1249.965288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_timeout -r 1 [ 1250.016636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1250.066622] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1250.306901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1250.329244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1250.356571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1250.389394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1250.433968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1250.520412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1250.559287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1250.584991] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1250.618861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1250.667176] LNet: 63075:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1250.704898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1250.732076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1250.763467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1250.810157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1250.864312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1250.938568] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1250.997315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1251.313173] LNet: 63116:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1251.317823] LNetError: 63116:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1251.337702] LNet: Removed LNI 192.168.201.17@tcp [ 1251.705766] Key type .llcrypt unregistered [ 1251.707219] Key type ._llcrypt unregistered [ 1270.966825] Key type ._llcrypt registered [ 1270.969479] Key type .llcrypt registered [ 1272.627633] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 20:00:07 (1788652807) [ 1273.728426] Key type .llcrypt unregistered [ 1273.730764] Key type ._llcrypt unregistered [ 1281.476803] Key type ._llcrypt registered [ 1281.478267] Key type .llcrypt registered [ 1281.835666] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1281.892788] alg: No test for adler32 (adler32-zlib) [ 1282.899305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1282.950147] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1291.879322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1301.182616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1301.254896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1301.274423] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1302.870657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_dropped -r 1 [ 1302.905094] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e remote_dropped -r 1 [ 1302.931254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_dropped -r 1 [ 1302.950253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e remote_dropped -r 1 [ 1302.984773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e remote_dropped -r 1 [ 1303.038551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e remote_dropped -r 1 [ 1303.076649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e remote_dropped -r 1 [ 1303.108633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e remote_dropped -r 1 [ 1303.129986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1303.144109] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1303.147274] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.117@tcp [ 1303.341257] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1303.363312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1303.386593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1303.414616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1303.436702] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1303.458645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1303.490471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1303.523940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1303.558672] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1303.582188] LNet: 64582:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1303.598113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1303.656531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1303.695939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1303.733618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1303.756542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1303.785360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1303.836657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1303.877818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1304.395182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_error -r 1 [ 1304.434192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e remote_error -r 1 [ 1304.490499] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_error -r 1 [ 1304.527872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e remote_error -r 1 [ 1304.567397] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e remote_error -r 1 [ 1304.609908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e remote_error -r 1 [ 1304.639882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e remote_error -r 1 [ 1304.673473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e remote_error -r 1 [ 1304.716691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1304.734544] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1304.739328] LNet: Skipped 2 previous similar messages [ 1304.926225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1304.951716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1304.968603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1304.999452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1305.039219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1305.067403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1305.100199] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1305.129638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1305.154920] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1305.167599] LNet: 64708:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1305.182848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1305.212660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1305.255899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1305.278938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1305.302755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1305.325206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1305.349933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1305.380071] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1305.857115] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_timeout -r 1 [ 1305.889184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e remote_timeout -r 1 [ 1305.920313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_timeout -r 1 [ 1305.954752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e remote_timeout -r 1 [ 1305.987825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e remote_timeout -r 1 [ 1306.009430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e remote_timeout -r 1 [ 1306.030874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e remote_timeout -r 1 [ 1306.058663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e remote_timeout -r 1 [ 1306.096174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1306.118216] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1306.123548] LNet: Skipped 1 previous similar message [ 1306.333296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1306.362744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1306.389087] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1306.429665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1306.464809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1306.497335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1306.528874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1306.563971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1306.595379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1306.617957] LNet: 64833:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1306.640480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1306.668174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1306.715441] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1306.754649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1306.808409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1306.852702] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1306.889453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1306.945840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1311.086671] LNet: 64911:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1311.091931] LNetError: 64911:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1311.100459] LNet: Removed LNI 192.168.201.17@tcp [ 1311.120516] LNet: Removed LNI 192.168.201.17@tcp1 [ 1311.513714] Key type .llcrypt unregistered [ 1311.517264] Key type ._llcrypt unregistered [ 1327.729438] Key type ._llcrypt registered [ 1327.734077] Key type .llcrypt registered [ 1328.859615] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 20:01:04 (1788652864) [ 1330.172728] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_208 kernel ip2nets deprecated in 2.17.53 [ 1331.969126] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 20:01:07 (1788652867) [ 1332.826898] Key type .llcrypt unregistered [ 1332.830888] Key type ._llcrypt unregistered [ 1338.481659] Key type ._llcrypt registered [ 1338.484453] Key type .llcrypt registered [ 1338.677451] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1338.716398] alg: No test for adler32 (adler32-zlib) [ 1339.696927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1339.727497] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1346.889619] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1347.786340] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e network_timeout -r 1 [ 1347.858326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e network_timeout -r 1 [ 1347.902832] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1347.923857] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1347.935526] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1348.222673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1348.260580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1348.314864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1348.363852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1348.427964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1348.484558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1348.533930] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1348.549478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1348.568330] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1348.595662] LNet: 66466:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1348.616852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1348.653778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1348.702454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1348.729437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1348.773761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1348.805590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1349.021605] LNet: 66505:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1349.026766] LNetError: 66505:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1349.041977] LNet: Removed LNI 192.168.201.17@tcp [ 1349.387779] Key type .llcrypt unregistered [ 1349.389911] Key type ._llcrypt unregistered [ 1369.928594] Key type ._llcrypt registered [ 1369.930508] Key type .llcrypt registered [ 1370.353398] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1370.388721] alg: No test for adler32 (adler32-zlib) [ 1371.373820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1371.437154] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1378.303357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1389.328411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1389.357889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1389.371830] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1391.237190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e network_timeout -r 1 [ 1391.278582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e network_timeout -r 1 [ 1391.323955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e network_timeout -r 1 [ 1391.364436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e network_timeout -r 1 [ 1391.402846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e network_timeout -r 1 [ 1391.441077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e network_timeout -r 1 [ 1391.476899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e network_timeout -r 1 [ 1391.507526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e network_timeout -r 1 [ 1391.545435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1391.566132] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1391.861526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1391.942263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1391.975373] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1391.977971] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.117@tcp [ 1392.001767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1392.039163] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1392.087133] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1392.139631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1392.173881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1392.209153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1392.268706] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1392.296640] LNet: 67393:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1392.312796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1392.339974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1392.365607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1392.392853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1392.430206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1392.467471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1396.738667] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1397.126845] LNet: 67467:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1397.134139] LNetError: 67467:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1397.142906] LNet: Removed LNI 192.168.201.17@tcp [ 1397.155736] LNet: Removed LNI 192.168.201.17@tcp1 [ 1397.617500] Key type .llcrypt unregistered [ 1397.620277] Key type ._llcrypt unregistered [ 1416.283445] Key type ._llcrypt registered [ 1416.285595] Key type .llcrypt registered [ 1417.623331] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 20:02:32 (1788652952) [ 1418.087723] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1418.117894] alg: No test for adler32 (adler32-zlib) [ 1419.004654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1419.052535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1419.068296] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1419.754229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1419.770467] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1420.717637] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 1420.780099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 1420.822492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 1420.856447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 1420.893363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1420.929227] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1420.958185] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.17@tcp [ 1420.988899] LNetError: 68566:0:(api-ni.c:4478:lnet_ni_set_healthv()) manually adding local NI 192.168.201.17@tcp to recovery [ 1421.089986] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1421.095312] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1423.136498] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1423.453685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1427.231564] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1428.144803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1428.177863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1428.237720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1428.256935] LNet: 68674:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1428.266087] LNetError: 68674:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1428.270071] LNet: Removed LNI 192.168.201.17@tcp [ 1428.276975] LNet: Removed LNI 192.168.201.17@tcp1 [ 1428.312546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1428.377528] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1428.402313] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1429.043570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1429.055630] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1429.688762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 1429.731832] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1429.783130] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 1429.827855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 1429.868891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 1429.897611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 1429.929486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1429.966314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.17@tcp [ 1429.983836] LNetError: 68946:0:(api-ni.c:4478:lnet_ni_set_healthv()) manually adding local NI 192.168.201.17@tcp to recovery [ 1430.367579] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1433.454578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1436.511496] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1436.515580] LNet: Skipped 1 previous similar message [ 1441.322262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1441.344983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1441.369523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1447.429573] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 20:03:02 (1788652982) [ 1447.520260] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1447.548956] LNet: 69654:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1447.556554] LNetError: 69654:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1447.562518] LNetError: 69654:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1447.582211] LNet: Removed LNI 192.168.201.17@tcp [ 1447.601058] LNet: Removed LNI 192.168.201.17@tcp1 [ 1447.670958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1447.799185] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1447.826827] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1448.345521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1448.365134] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1449.166110] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 1449.223581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 1449.248576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 1449.272902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 1449.301634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1449.327458] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1449.351675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.17@tcp --health 0 [ 1449.446898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1449.823161] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.17@tcp [ 1450.850305] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1450.858573] LNet: Skipped 1 previous similar message [ 1465.002306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1465.120983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1465.144179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1465.197238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.17@tcp --health 500 [ 1465.301636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1471.922378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1471.972304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1471.994699] LNet: 70154:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1472.005978] LNetError: 70154:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1472.019420] LNetError: 70154:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1472.035812] LNet: Removed LNI 192.168.201.17@tcp [ 1472.047179] LNet: Removed LNI 192.168.201.17@tcp1 [ 1472.082585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1472.142806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1472.161836] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1472.978674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1473.046627] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1473.695949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 1473.732680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1473.785486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 1473.807392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 1473.841502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 1473.894547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1473.931286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1473.984261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.17@tcp --health 0 [ 1475.233150] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1475.247592] LNet: Skipped 3 previous similar messages [ 1476.623971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1485.795786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1485.839271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1485.878396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1492.789391] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 20:03:47 (1788653027) [ 1493.356664] LNet: 71201:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1493.367347] LNetError: 71201:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1493.374958] LNetError: 71201:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1493.385450] LNet: Removed LNI 192.168.201.17@tcp [ 1493.397973] LNet: Removed LNI 192.168.201.17@tcp1 [ 1494.019849] Key type .llcrypt unregistered [ 1494.023716] Key type ._llcrypt unregistered [ 1500.768836] Key type ._llcrypt registered [ 1500.771595] Key type .llcrypt registered [ 1500.976750] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1501.014284] alg: No test for adler32 (adler32-zlib) [ 1501.944562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1501.984257] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1510.477757] Lustre: DEBUG MARKER: Initial discovery [ 1510.615871] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.117@tcp [ 1512.980340] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag [ 1513.194367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 -e local_error [ 1513.227645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.117@tcp [ 1513.245464] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1513.265890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1513.296609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.201.117@tcp -r 1 -m GET -l 3 [ 1513.334928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.201.117@tcp -r 1 -m GET -e local_error [ 1513.370281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d 192.168.201.117@tcp -r 1 -m PUT -l 6 [ 1514.272217] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1515.117300] Lustre: DEBUG MARKER: Force oleg117-client.virtnet to discover 192.168.201.117@tcp (in background) [ 1515.211065] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.117@tcp [ 1516.514837] Lustre: DEBUG MARKER: Force oleg117-server to discover 192.168.201.17@tcp [ 1518.241157] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1519.479734] Lustre: DEBUG MARKER: Wait for 71951 [ 1521.044622] Lustre: DEBUG MARKER: Finished wait on 71951 [ 1521.534725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay del -a [ 1521.623657] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1521.647839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1522.118844] LNet: 72410:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1522.125208] LNetError: 72410:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1522.144039] LNet: Removed LNI 192.168.201.17@tcp [ 1522.584670] Key type .llcrypt unregistered [ 1522.586135] Key type ._llcrypt unregistered [ 1537.771114] Key type ._llcrypt registered [ 1537.772529] Key type .llcrypt registered [ 1538.896962] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 20:04:34 (1788653074) [ 1539.441413] Key type .llcrypt unregistered [ 1539.444873] Key type ._llcrypt unregistered [ 1546.088713] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1546.210724] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1546.223261] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1546.231553] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1548.801560] Key type ._llcrypt registered [ 1548.804243] Key type .llcrypt registered [ 1548.945115] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1548.954489] alg: No test for adler32 (adler32-zlib) [ 1549.853702] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1549.892824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1549.913662] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1550.470088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1550.486246] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1557.417917] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 20:04:52 (1788653092) [ 1557.488212] LNet: 74485:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1557.496204] LNetError: 74485:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1557.509904] LNet: Removed LNI 10.1.2.3@tcp [ 1557.517733] LNet: Removed LNI 192.168.201.17@tcp [ 1558.065610] Key type .llcrypt unregistered [ 1558.068285] Key type ._llcrypt unregistered [ 1564.343394] Key type ._llcrypt registered [ 1564.345802] Key type .llcrypt registered [ 1564.493848] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1564.527589] alg: No test for adler32 (adler32-zlib) [ 1565.471473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1565.557386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1565.580919] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1566.338572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1566.356410] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1567.319829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 10.1.2.3@tcp 192.168.201.17@tcp [ 1568.358906] LNetError: Unexpected error -22 connecting to 192.168.201.17@tcp at host 192.168.201.17:988 [ 1572.791708] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 20:05:08 (1788653108) [ 1572.997082] LNet: 75622:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1573.013987] LNetError: 75622:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1573.028946] LNet: Removed LNI 10.1.2.3@tcp [ 1573.045574] LNet: Removed LNI 192.168.201.17@tcp [ 1573.739775] Key type .llcrypt unregistered [ 1573.747989] Key type ._llcrypt unregistered [ 1579.834247] Key type ._llcrypt registered [ 1579.839901] Key type .llcrypt registered [ 1579.994517] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1580.014503] alg: No test for adler32 (adler32-zlib) [ 1580.928024] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1580.978136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1580.999135] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1581.624310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1581.645359] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 1582.535627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 192.168.201.17@tcp1 --nid 192.168.201.17@tcp2 [ 1586.734353] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp1 192.168.201.17@tcp2 [ 1586.773872] LNetError: 76454:0:(lib-move.c:2094:lnet_handle_find_routed_path()) Unable to send message from 192.168.201.17@tcp1 to 192.168.201.17@tcp2 - Route table may be misconfigured [ 1586.802557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp2 192.168.201.17@tcp1 [ 1593.160501] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 20:05:28 (1788653128) [ 1593.402308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1593.437530] LNet: 77027:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1593.446672] LNetError: 77027:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1593.465408] LNet: Removed LNI 192.168.201.17@tcp1 [ 1594.552526] LNet: Removed LNI 192.168.201.17@tcp2 [ 1594.602100] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1594.772504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1594.826790] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1595.674851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1595.702343] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1596.629648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 1596.660302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.17@tcp [ 1596.695863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp1 192.168.201.17@tcp1 [ 1596.734650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.17@tcp -d 192.168.201.17@tcp -e network_timeout [ 1596.755702] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -e network_timeout [ 1596.775967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -e network_timeout [ 1596.800646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -e network_timeout [ 1596.828958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1596.855351] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1596.877894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1596.989556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1597.059674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1597.728561] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1603.047822] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 20:05:37 (1788653137) [ 1603.273621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1603.308804] LNet: 77882:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1603.325378] LNetError: 77882:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1603.334893] LNetError: 77882:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1603.342403] LNet: Removed LNI 192.168.201.17@tcp [ 1604.402216] LNet: Removed LNI 192.168.201.17@tcp1 [ 1604.483336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1604.628516] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 0@lo [ 1605.017280] LNet: 77926:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1605.564524] Key type .llcrypt unregistered [ 1605.566283] Key type ._llcrypt unregistered [ 1620.377396] Key type ._llcrypt registered [ 1620.379641] Key type .llcrypt registered [ 1622.432858] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 20:05:56 (1788653156) [ 1622.998693] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1623.027668] alg: No test for adler32 (adler32-zlib) [ 1623.910387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1624.983154] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces [ 1626.707584] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 20:06:01 (1788653161) [ 1627.247874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1627.270777] LNet: 79021:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1627.288803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1627.326933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1627.340831] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1627.878129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1627.921062] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1629.100992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1629.172572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp1 [ 1629.267975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp1 [ 1635.567215] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 20:06:10 (1788653170) [ 1637.727411] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_220 Need at least 2 remote nodes found oleg117-server [ 1639.768294] Lustre: DEBUG MARKER: == sanity-lnet test 221: Add routes w/DD disabled - check aliveness ========================================================== 20:06:14 (1788653174) [ 1642.366418] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_221 Need at least 2 remote nodes found oleg117-server [ 1643.964606] Lustre: DEBUG MARKER: == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 20:06:19 (1788653179) [ 1645.821352] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_222 Need at least 2 remote nodes found oleg117-server [ 1647.875647] Lustre: DEBUG MARKER: == sanity-lnet test 223: Check avoid_asym_router_failure=1 w/DD disabled ========================================================== 20:06:23 (1788653183) [ 1650.053668] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_223 Need at least 2 remote nodes found oleg117-server [ 1652.954693] Lustre: DEBUG MARKER: == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 20:06:27 (1788653187) [ 1654.825612] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_224 Need at least 2 remote nodes found oleg117-server [ 1656.891996] Lustre: DEBUG MARKER: == sanity-lnet test 225: Check avoid_asym_router_failure=0 w/DD disabled ========================================================== 20:06:31 (1788653191) [ 1659.324848] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_225 Need at least 2 remote nodes found oleg117-server [ 1661.001503] Lustre: DEBUG MARKER: == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 20:06:36 (1788653196) [ 1663.519457] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_226 Need at least 3 remote nodes found oleg117-server [ 1665.415907] Lustre: DEBUG MARKER: == sanity-lnet test 227: Check router peer health w/DD disabled ========================================================== 20:06:40 (1788653200) [ 1667.428773] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_227 Need at least 3 remote nodes found oleg117-server [ 1669.379985] Lustre: DEBUG MARKER: == sanity-lnet test 228: Routes should stay up when health is decremented ========================================================== 20:06:44 (1788653204) [ 1670.988827] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_228 Need at least 3 remote nodes found oleg117-server [ 1672.762660] Lustre: DEBUG MARKER: == sanity-lnet test 230: Test setting conns-per-peer ===== 20:06:47 (1788653207) [ 1672.956120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1673.011519] LNet: 82899:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1673.026120] LNetError: 82899:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1673.050572] LNet: Removed LNI 192.168.201.17@tcp [ 1674.097546] LNet: Removed LNI 192.168.201.17@tcp1 [ 1674.146915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1674.257378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1674.292517] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1675.134246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 4 [ 1675.245942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1675.502821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1675.536298] LNet: 83047:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1675.547671] LNetError: 83047:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1675.554115] LNetError: 83047:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1676.589952] LNet: Removed LNI 192.168.201.17@tcp [ 1676.621081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1676.678147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 5 [ 1676.716122] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1677.562931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1677.723857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1677.759443] LNet: 83192:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1677.766198] LNetError: 83192:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1678.821545] LNet: Removed LNI 192.168.201.17@tcp [ 1678.867229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1678.928582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1678.971260] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1679.675124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 6 [ 1679.730532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1679.865660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1679.876952] LNetError: 83340:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1679.889565] LNet: Removed LNI 192.168.201.17@tcp [ 1679.903677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1679.953666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 7 [ 1679.984297] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1680.827609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1681.013918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1682.087797] LNet: Removed LNI 192.168.201.17@tcp [ 1682.142399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1682.225132] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1682.254413] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1683.171264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 8 [ 1683.238489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1683.559462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1683.652838] LNet: 83632:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1683.679263] LNet: 83632:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 1684.775670] LNet: Removed LNI 192.168.201.17@tcp [ 1684.850714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1684.959922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 9 [ 1685.025065] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1685.801032] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1685.985874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1686.002315] LNetError: 83778:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1686.006172] LNetError: 83778:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 1687.080819] LNet: Removed LNI 192.168.201.17@tcp [ 1687.099402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1687.190445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1687.213092] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1687.841283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 10 [ 1687.904528] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1688.136607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1689.193957] LNet: Removed LNI 192.168.201.17@tcp [ 1689.221460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1689.286823] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 11 [ 1689.326592] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1689.984974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1690.227342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1691.321987] LNet: Removed LNI 192.168.201.17@tcp [ 1691.343876] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1691.400472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1691.423565] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1692.092940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 12 [ 1692.213704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1692.541320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1692.553281] LNet: 84218:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1692.557019] LNet: 84218:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 1693.605711] LNet: Removed LNI 192.168.201.17@tcp [ 1693.641680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1693.714923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 13 [ 1693.751088] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1694.676930] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1694.961800] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1695.014641] LNetError: 84363:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1695.019482] LNetError: 84363:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 1696.041358] LNet: Removed LNI 192.168.201.17@tcp [ 1696.076843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1696.149953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1696.176254] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1696.714745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 14 [ 1696.780774] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1696.880366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1698.024153] LNet: Removed LNI 192.168.201.17@tcp [ 1698.106574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1698.165656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 15 [ 1698.185377] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1698.879692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1699.140535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1700.200656] LNet: Removed LNI 192.168.201.17@tcp [ 1700.227934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1700.332862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1700.361963] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1701.119847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 128 [ 1701.196254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1701.236867] LNet: Removed LNI 192.168.201.17@tcp [ 1701.250570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1701.285675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1701.299668] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1702.115088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer -1 [ 1702.176836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1702.199587] LNet: Removed LNI 192.168.201.17@tcp [ 1702.229859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1702.265870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 128 [ 1702.356857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1702.439502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1702.494696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 1000000000000000000000000 [ 1702.611776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1702.633716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1702.658656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer foo [ 1702.697717] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1702.728637] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1702.769656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 8 [ 1702.796846] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1703.289549] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1703.312817] LNet: Removed LNI 192.168.201.17@tcp [ 1703.334923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1703.412668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 1712.303205] Lustre: DEBUG MARKER: == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 20:07:26 (1788653246) [ 1712.465506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1712.492566] LNet: 85670:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1712.500655] LNet: 85670:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 9 previous similar messages [ 1712.532970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1712.642272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 [ 1712.705838] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/180] [ 1714.891912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1714.932944] LNetError: 85816:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1714.945957] LNetError: 85816:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 1714.995900] LNet: Removed LNI 192.168.201.17@tcp231 [ 1715.047527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1715.138102] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1715.168166] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/180] [ 1715.183460] LNet: 85824:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1716.132836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1716.174885] LNet: Removed LNI 192.168.201.17@tcp231 [ 1716.200887] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=-1 [ 1716.246743] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/180] [ 1717.591271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1717.630509] LNet: Removed LNI 192.168.201.17@tcp231 [ 1717.656784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1717.764207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1717.800770] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/0] [ 1717.806179] LNet: 85977:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1718.366331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1718.439600] LNet: Removed LNI 192.168.201.17@tcp231 [ 1718.457489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=0 [ 1718.530716] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/0] [ 1720.096361] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1720.167923] LNet: Removed LNI 192.168.201.17@tcp231 [ 1720.196653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1720.294237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1720.331039] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/60] [ 1720.336687] LNet: 86128:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1720.983295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1721.034482] LNet: Removed LNI 192.168.201.17@tcp231 [ 1721.049900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=60 [ 1721.099141] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/60] [ 1722.498090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1722.549210] LNet: Removed LNI 192.168.201.17@tcp231 [ 1722.593526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1722.687450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1722.719522] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/180] [ 1722.748286] LNet: 86279:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1723.219517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1723.261218] LNet: Removed LNI 192.168.201.17@tcp231 [ 1723.281578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=180 [ 1723.330446] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/180] [ 1732.318042] Lustre: DEBUG MARKER: == sanity-lnet test 232: Test setting ToS value ========== 20:07:47 (1788653267) [ 1732.772662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1732.820858] LNet: Removed LNI 192.168.201.17@tcp231 [ 1732.855850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1732.985910] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 [ 1733.039533] LNet: Added LNI 192.168.201.17@tcp232 [8/256/0/180] [ 1733.871164] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1733.908660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1733.929372] LNet: Removed LNI 192.168.201.17@tcp232 [ 1733.951988] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos -1 [ 1733.973788] LNet: Added LNI 192.168.201.17@tcp232 [8/256/0/180] [ 1735.100946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1735.129795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1735.160717] LNet: Removed LNI 192.168.201.17@tcp232 [ 1735.171911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 104 [ 1735.191051] LNet: Added LNI 192.168.201.17@tcp232 [8/256/0/180] [ 1735.996670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1736.034693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1736.071163] LNet: Removed LNI 192.168.201.17@tcp232 [ 1736.088767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 106 [ 1736.120464] LNet: Added LNI 192.168.201.17@tcp232 [8/256/0/180] [ 1736.874632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1736.906716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1736.938064] LNet: Removed LNI 192.168.201.17@tcp232 [ 1736.959034] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 300 [ 1744.046528] Lustre: DEBUG MARKER: == sanity-lnet test 233: Check for successful resends ==== 20:07:58 (1788653278) [ 1745.193156] Key type .llcrypt unregistered [ 1745.195644] Key type ._llcrypt unregistered [ 1753.405232] Key type ._llcrypt registered [ 1753.406850] Key type .llcrypt registered [ 1753.764955] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1753.791698] alg: No test for adler32 (adler32-zlib) [ 1754.724538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1754.749670] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1763.128495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1774.488801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1774.573682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1774.618927] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1776.516532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_interrupt -r 2 [ 1776.566830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_interrupt -r 2 [ 1776.615241] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_interrupt -r 2 [ 1776.660542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_interrupt -r 2 [ 1776.715248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_interrupt -r 2 [ 1776.775180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_interrupt -r 2 [ 1776.827444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_interrupt -r 2 [ 1776.883966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_interrupt -r 2 [ 1776.934852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1776.979122] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1776.991293] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1777.252259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1777.281460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1777.318572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1777.348981] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1777.382406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1777.443323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1777.497294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1777.538879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1777.567408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1777.590214] LNet: 88780:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1777.614709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1777.641935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1777.697628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1777.725434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1777.752364] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1777.781551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1778.223293] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_dropped -r 2 [ 1778.264722] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_dropped -r 2 [ 1778.315528] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_dropped -r 2 [ 1778.370890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_dropped -r 2 [ 1778.444140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_dropped -r 2 [ 1778.482432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_dropped -r 2 [ 1778.538389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_dropped -r 2 [ 1778.604882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_dropped -r 2 [ 1778.661130] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1778.732740] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1779.110697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1779.138699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1779.173029] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1779.220980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1779.270756] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1779.351952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1779.421120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1779.483926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1779.569094] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1779.618790] LNet: 88903:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1779.666972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1779.775852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1779.843485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1779.908698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1779.960501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1780.029513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1781.152593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_aborted -r 2 [ 1781.199562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_aborted -r 2 [ 1781.250861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_aborted -r 2 [ 1781.315221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_aborted -r 2 [ 1781.386207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_aborted -r 2 [ 1781.448975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_aborted -r 2 [ 1781.487343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_aborted -r 2 [ 1781.547047] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_aborted -r 2 [ 1781.589151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1781.614404] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1781.915113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1781.982148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1782.051975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1782.112884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1782.162112] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1782.193887] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1782.230361] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1782.259979] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1782.291936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1782.314377] LNet: 89025:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1782.334860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1782.421908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1782.502274] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1782.554346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1782.595408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1782.625813] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1783.364000] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_no_route -r 2 [ 1783.395356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_no_route -r 2 [ 1783.436176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_no_route -r 2 [ 1783.490992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_no_route -r 2 [ 1783.556897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_no_route -r 2 [ 1783.595689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_no_route -r 2 [ 1783.628877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_no_route -r 2 [ 1783.665892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_no_route -r 2 [ 1783.703317] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1783.717294] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1783.996529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1784.062350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1784.102898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1784.129349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1784.154443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1784.183227] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1784.212568] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1784.260749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1784.294178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1784.333520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1784.381694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1784.425358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1784.471535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1784.531253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1784.560839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1785.303903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_timeout -r 2 [ 1785.340642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_timeout -r 2 [ 1785.390240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_timeout -r 2 [ 1785.427714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_timeout -r 2 [ 1785.483179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_timeout -r 2 [ 1785.538849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_timeout -r 2 [ 1785.581749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_timeout -r 2 [ 1785.613355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_timeout -r 2 [ 1785.642187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1786.011918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1786.064773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1786.129398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1786.186642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1786.245071] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1786.288790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1786.348395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1786.409506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1786.454275] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1786.467701] LNet: 89270:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1786.472388] LNet: 89270:0:(api-ni.c:301:recovery_interval_set()) Skipped 1 previous similar message [ 1786.485677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1786.530968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1786.572940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1786.607374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1786.644605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1786.686271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1787.439432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_dropped -r 2 [ 1787.492113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e remote_dropped -r 2 [ 1787.554781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_dropped -r 2 [ 1787.612711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e remote_dropped -r 2 [ 1787.652326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e remote_dropped -r 2 [ 1787.697393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e remote_dropped -r 2 [ 1787.730463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e remote_dropped -r 2 [ 1787.772286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e remote_dropped -r 2 [ 1787.813564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1787.834716] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1787.840255] LNet: Skipped 1 previous similar message [ 1787.848029] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.117@tcp [ 1788.076292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1788.116636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1788.170852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1788.209295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1788.239618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1788.296694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1788.337129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1788.381887] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1788.444803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1788.476757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1788.514254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1788.552562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1788.599634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1788.638480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1788.672336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1794.884988] LNet: 89466:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1794.892613] LNetError: 89466:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1795.935144] LNet: Removed LNI 192.168.201.17@tcp [ 1795.947132] LNetError: 89466:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1795.972771] LNet: Removed LNI 192.168.201.17@tcp1 [ 1796.564890] Key type .llcrypt unregistered [ 1796.566622] Key type ._llcrypt unregistered [ 1815.149439] Key type ._llcrypt registered [ 1815.151866] Key type .llcrypt registered [ 1816.532701] Lustre: DEBUG MARKER: == sanity-lnet test 234: Verify NI status changes with manual health value transitions ========================================================== 20:09:11 (1788653351) [ 1816.980563] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1816.995740] alg: No test for adler32 (adler32-zlib) [ 1818.055285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1818.164449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1818.201850] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1819.101927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 0 [ 1819.137487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.17@tcp [ 1819.158461] LNetError: 90443:0:(api-ni.c:4478:lnet_ni_set_healthv()) manually adding local NI 192.168.201.17@tcp to recovery [ 1819.168357] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1819.259477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --nid 192.168.201.17@tcp [ 1819.356835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1826.606749] Lustre: DEBUG MARKER: == sanity-lnet test 235: Verify NI status DOWN when health reaches 0 after failures ========================================================== 20:09:21 (1788653361) [ 1826.697253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1826.738616] LNet: 91026:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1826.746205] LNetError: 91026:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1826.768157] LNet: Removed LNI 192.168.201.17@tcp [ 1826.805222] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1826.940152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1826.965253] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1827.657662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1827.700534] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1828.971423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 100 --nid 192.168.201.17@tcp [ 1828.993577] LNetError: 91300:0:(api-ni.c:4478:lnet_ni_set_healthv()) manually adding local NI 192.168.201.17@tcp to recovery [ 1829.956402] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1830.395619] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 900 --nid 192.168.201.17@tcp [ 1840.339684] Lustre: DEBUG MARKER: == sanity-lnet test 236: Local NI state propagates to routers ========================================================== 20:09:35 (1788653375) [ 1842.678653] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_236 Need at least 2 remote nodes found oleg117-server [ 1844.821149] Lustre: DEBUG MARKER: == sanity-lnet test 237: Check MR forwarding feature ===== 20:09:39 (1788653379) [ 1847.517591] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_237 Need at least 2 remote nodes found oleg117-server [ 1849.391674] Lustre: DEBUG MARKER: == sanity-lnet test 238: Check MR forwarding feature skipped for non-MR source ========================================================== 20:09:44 (1788653384) [ 1851.481226] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_238 Need at least 2 remote nodes found oleg117-server [ 1853.623276] Lustre: DEBUG MARKER: == sanity-lnet test 241: Check setting LND timeout value via lnetctl updates tunables ========================================================== 20:09:48 (1788653388) [ 1854.090452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1854.116607] LNet: 92932:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1854.121546] LNetError: 92932:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1854.126961] LNet: Removed LNI 192.168.201.17@tcp [ 1854.139538] LNet: Removed LNI 192.168.201.17@tcp1 [ 1854.160705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1854.229755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1854.261520] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1854.942833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set -h [ 1854.999954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 51 [ 1855.033745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp -v [ 1855.120135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 0 [ 1855.158650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp -v [ 1855.229172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 50 [ 1862.835669] Lustre: DEBUG MARKER: == sanity-lnet test 245: Check read-only params do not appear in export --backup ========================================================== 20:09:57 (1788653397) [ 1862.960166] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1862.991539] LNet: 93659:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1863.002614] LNetError: 93659:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1863.005627] LNetError: 93659:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1863.034708] LNet: Removed LNI 192.168.201.17@tcp [ 1863.078191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1863.126458] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1863.146706] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1870.900680] Lustre: DEBUG MARKER: == sanity-lnet test 250: test that linux routes are added ========================================================== 20:10:05 (1788653405) [ 1871.079345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1871.147708] LNet: 94365:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1871.158241] LNetError: 94365:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1871.178015] LNet: Removed LNI 192.168.201.17@tcp [ 1871.212193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1871.287290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1871.350952] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1880.842945] Lustre: DEBUG MARKER: == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 20:10:15 (1788653415) [ 1882.803496] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_251 Need kfi NETTYPE [ 1884.689501] Lustre: DEBUG MARKER: == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 20:10:19 (1788653419) [ 1885.803332] LNet: 95403:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1885.813249] LNetError: 95403:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1885.829448] LNet: Removed LNI 192.168.201.17@tcp [ 1886.461682] Key type .llcrypt unregistered [ 1886.470553] Key type ._llcrypt unregistered [ 1895.148907] Key type ._llcrypt registered [ 1895.161450] Key type .llcrypt registered [ 1895.482505] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1895.497848] alg: No test for adler32 (adler32-zlib) [ 1896.572446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1896.612542] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1905.171362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1909.844842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --timeout 15 192.168.201.117@tcp [ 1910.250642] LNet: 95818:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1910.256407] LNetError: 95818:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1910.279532] LNet: Removed LNI 192.168.201.17@tcp [ 1910.747699] Key type .llcrypt unregistered [ 1910.752396] Key type ._llcrypt unregistered [ 1923.039986] Key type ._llcrypt registered [ 1923.044784] Key type .llcrypt registered [ 1924.276652] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253 [ 1926.557816] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254 [ 1928.246754] Lustre: DEBUG MARKER: == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 20:11:03 (1788653463) [ 1929.900838] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_255 kernel ip2nets deprecated in 2.17.53 [ 1931.402863] Lustre: DEBUG MARKER: == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 20:11:06 (1788653466) [ 1933.190340] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_256 Need at least 2 remote nodes found oleg117-server [ 1935.121782] Lustre: DEBUG MARKER: == sanity-lnet test 257: Test graceful router shutdown/startup ========================================================== 20:11:10 (1788653470) [ 1937.081180] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_257 Need at least 3 remote nodes found oleg117-server [ 1938.823085] Lustre: DEBUG MARKER: == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 20:11:13 (1788653473) [ 1940.476189] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1940.643975] alg: No test for adler32 (adler32-zlib) [ 1941.811421] Lustre: Lustre: Build Version: 2.17.58_2_g557ce73 [ 1942.087545] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1943.783275] Key type lgssc registered [ 1945.302972] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1966.648030] Key type lgssc unregistered [ 1966.900833] LNet: 98991:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1966.905886] LNetError: 98991:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1966.926613] LNet: Removed LNI 192.168.201.17@tcp [ 1967.488581] Key type .llcrypt unregistered [ 1967.490614] Key type ._llcrypt unregistered [ 1984.056882] Key type ._llcrypt registered [ 1984.059816] Key type .llcrypt registered [ 1985.517773] Lustre: DEBUG MARKER: == sanity-lnet test 265: Import of invalid CPT should fail ========================================================== 20:12:00 (1788653520) [ 1985.999290] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1986.013733] alg: No test for adler32 (adler32-zlib) [ 1987.092350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1987.149974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 1987.208491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 1987.258847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 1987.310809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 1994.200809] Lustre: DEBUG MARKER: == sanity-lnet test 266: Validate CPT parsing in network sequence ========================================================== 20:12:09 (1788653529) [ 1994.405614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1994.448303] LNet: 100403:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1994.510305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1994.611874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1994.649912] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1995.694730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1995.735226] LNet: 100548:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1995.740204] LNetError: 100548:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1995.762740] LNet: Removed LNI 192.168.201.17@tcp [ 1995.784760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1995.853645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 1995.886289] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1996.513321] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1996.545870] LNetError: 100576:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1996.560700] LNet: Removed LNI 192.168.201.17@tcp [ 1996.581795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1996.654757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 1996.677797] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2003.761632] Lustre: DEBUG MARKER: == sanity-lnet test 270: Test NID metadata =============== 20:12:18 (1788653538) [ 2003.939530] LNet: 101161:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2003.954389] LNet: 101161:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 2003.967598] LNetError: 101161:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2003.984649] LNet: Removed LNI 192.168.201.17@tcp [ 2004.592804] Key type .llcrypt unregistered [ 2004.594576] Key type ._llcrypt unregistered [ 2011.805202] Key type ._llcrypt registered [ 2011.807251] Key type .llcrypt registered [ 2012.102288] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2012.129371] alg: No test for adler32 (adler32-zlib) [ 2013.081038] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2013.203796] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2013.206767] LNet: *** cfs_fail_loc=e003, val=100*** [ 2013.998944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2014.018316] LNet: *** cfs_fail_loc=e003, val=100*** [ 2014.021323] LNet: Skipped 2 previous similar messages [ 2014.026384] Ping reply received with mapping: [ 2014.027934] found metadata reply NI[192.168.201.17@tcp] [ 2014.030433] 11 c9 a8 c0 00 00 02 00 5a 5a 5a 5a 5a 5a 5a 5a [ 2014.032747] 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a [ 2014.035687] 5a 5a 5a 5a 5a 5a 5a 5a [ 2014.107374] LNet: 101562:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2014.111850] LNetError: 101562:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2015.142996] LNet: Removed LNI 192.168.201.17@tcp [ 2015.977693] Key type .llcrypt unregistered [ 2015.979555] Key type ._llcrypt unregistered [ 2030.507733] Key type ._llcrypt registered [ 2030.510378] Key type .llcrypt registered [ 2032.166686] Lustre: DEBUG MARKER: == sanity-lnet test 280: Don't panic when request_module fails ========================================================== 20:12:46 (1788653566) [ 2032.940543] Key type .llcrypt unregistered [ 2032.943329] Key type ._llcrypt unregistered [ 2041.359647] Key type ._llcrypt registered [ 2041.364258] Key type .llcrypt registered [ 2041.592550] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2041.605855] alg: No test for adler32 (adler32-zlib) [ 2042.962459] LNetError: 102601:0:(api-ni.c:2855:lnet_load_lnd()) Can't load LND tcp, module ksocklnd, rc=256 [ 2043.478748] Key type .llcrypt unregistered [ 2043.479975] Key type ._llcrypt unregistered [ 2049.161477] Key type ._llcrypt registered [ 2049.163589] Key type .llcrypt registered [ 2050.669413] Lustre: DEBUG MARKER: == sanity-lnet test 290: Check that lnet acceptor is using explicit address binding ========================================================== 20:13:05 (1788653585) [ 2051.426115] Key type .llcrypt unregistered [ 2051.439597] Key type ._llcrypt unregistered [ 2060.243480] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2060.254432] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2062.711360] Key type ._llcrypt registered [ 2062.717459] Key type .llcrypt registered [ 2062.891525] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2062.908759] alg: No test for adler32 (adler32-zlib) [ 2063.901374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2064.014602] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2075.275535] Lustre: DEBUG MARKER: == sanity-lnet test 291: load lnet using custom accept port for bulk tcp connections ========================================================== 20:13:29 (1788653609) [ 2075.421964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2075.500775] LNet: 104371:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2075.506984] LNetError: 104371:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2075.533815] LNet: Removed LNI 10.1.2.3@tcp [ 2075.590179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2076.633926] Key type .llcrypt unregistered [ 2076.637833] Key type ._llcrypt unregistered [ 2083.675974] Key type ._llcrypt registered [ 2083.678672] Key type .llcrypt registered [ 2083.987223] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2084.011026] alg: No test for adler32 (adler32-zlib) [ 2084.992786] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2085.107168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2085.269956] LNet: 104661:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2085.275263] LNetError: 104661:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2085.287865] LNet: Removed LNI 192.168.201.17@tcp [ 2085.894458] Key type .llcrypt unregistered [ 2085.901926] Key type ._llcrypt unregistered [ 2098.111528] Key type ._llcrypt registered [ 2098.116217] Key type .llcrypt registered [ 2099.736838] Lustre: DEBUG MARKER: == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 20:13:54 (1788653634) [ 2100.427490] Key type .llcrypt unregistered [ 2100.431488] Key type ._llcrypt unregistered [ 2108.254804] Key type ._llcrypt registered [ 2108.261218] Key type .llcrypt registered [ 2108.702641] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2108.723327] alg: No test for adler32 (adler32-zlib) [ 2116.097367] Lustre: DEBUG MARKER: == sanity-lnet test 301: Check for dynamic adds of same/wrong interface (memory leak) ========================================================== 20:14:11 (1788653651) [ 2116.273249] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2116.323155] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2116.442656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2116.490974] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2117.273588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2117.308318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if ens2 [ 2117.329891] LNet: Removed LNI 192.168.201.17@tcp [ 2117.534358] LNet: 106424:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2117.907372] Key type .llcrypt unregistered [ 2117.908463] Key type ._llcrypt unregistered [ 2128.870411] Key type ._llcrypt registered [ 2128.872898] Key type .llcrypt registered [ 2130.145212] Lustre: DEBUG MARKER: == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 20:14:25 (1788653665) [ 2130.607645] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2130.632848] alg: No test for adler32 (adler32-zlib) [ 2131.594033] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2131.651421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2131.670633] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2132.597236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2132.629624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug peer --nid 192.168.201.17@tcp [ 2132.654879] LNet: Conn 0 [type, closing, crefcnt, srefcnt]: 2, 0, 1, 1 [ 2132.658192] LNet: Conn 0 rx [scheduled, ready, state]: 0, 0, 1 [ 2132.660916] LNet: Conn 0 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2132, 0, 0 [ 2132.669840] LNet: Conn 0 sched [nconns, cpt]: 4, 0 [ 2132.672282] LNet: Conn 1 [type, closing, crefcnt, srefcnt]: 3, 0, 1, 1 [ 2132.674538] LNet: Conn 1 rx [scheduled, ready, state]: 0, 0, 1 [ 2132.677193] LNet: Conn 1 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2132, 0, 0 [ 2132.681041] LNet: Conn 1 sched [nconns, cpt]: 4, 0 [ 2132.683348] LNet: Conn 2 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 2132.685917] LNet: Conn 2 rx [scheduled, ready, state]: 0, 0, 1 [ 2132.688250] LNet: Conn 2 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2132, 0, 2182 [ 2132.692366] LNet: Conn 2 sched [nconns, cpt]: 4, 0 [ 2132.694839] LNet: Conn 3 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 2132.697975] LNet: Conn 3 rx [scheduled, ready, state]: 0, 0, 1 [ 2132.700868] LNet: Conn 3 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2132, 0, 2182 [ 2132.704727] LNet: Conn 3 sched [nconns, cpt]: 4, 0 [ 2139.445157] Lustre: DEBUG MARKER: == sanity-lnet test 303: Check peer NI health after link down ========================================================== 20:14:34 (1788653674) [ 2140.009793] LNet: 107960:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2140.017840] LNetError: 107960:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2140.046908] LNet: Removed LNI 192.168.201.17@tcp [ 2140.770617] Key type .llcrypt unregistered [ 2140.772198] Key type ._llcrypt unregistered [ 2147.547710] Key type ._llcrypt registered [ 2147.550296] Key type .llcrypt registered [ 2147.910467] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2147.942639] alg: No test for adler32 (adler32-zlib) [ 2148.856747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2148.877912] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2158.799962] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 2169.437653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2169.534066] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2169.574687] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 2171.540517] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 2171.684459] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2171.702356] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2171.710506] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 2174.227989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp99 --if test1pg [ 2174.289252] LNet: Added LNI 10.1.2.3@tcp99 [8/256/0/180] [ 2187.490255] Lustre: DEBUG MARKER: == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 20:15:21 (1788653721) [ 2187.670914] LNet: 109467:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2187.683676] LNetError: 109467:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2187.698955] LNet: Removed LNI 192.168.201.17@tcp [ 2188.767306] LNet: Removed LNI 192.168.201.17@tcp1 [ 2188.769273] LNetError: 109467:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2188.780293] LNetError: 109467:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2188.795388] LNet: Removed LNI 10.1.2.3@tcp99 [ 2189.490819] Key type .llcrypt unregistered [ 2189.494936] Key type ._llcrypt unregistered [ 2196.825772] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2196.832971] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2199.798832] Key type ._llcrypt registered [ 2199.804422] Key type .llcrypt registered [ 2200.123022] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2200.169215] alg: No test for adler32 (adler32-zlib) [ 2201.336269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2201.496529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2201.542707] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2202.518394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2202.540540] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2204.962776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.1.2.3@tcp --lock_prim [ 2205.003317] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2205.050994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 10.1.2.3@tcp --state 1048577 [ 2205.075108] LNetError: 110217:0:(peer.c:2366:lnet_del_peer_ni()) peer 10.1.2.3@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 2205.124929] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2205.167299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2205.582871] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2205.722585] LNet: 110237:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2205.734620] LNetError: 110237:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2205.744942] LNet: Removed LNI 10.1.2.3@tcp [ 2206.767985] LNet: Removed LNI 192.168.201.17@tcp [ 2207.455738] Key type .llcrypt unregistered [ 2207.464357] Key type ._llcrypt unregistered [ 2219.498207] Key type ._llcrypt registered [ 2219.502477] Key type .llcrypt registered [ 2220.752754] Lustre: DEBUG MARKER: == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 20:15:55 (1788653755) [ 2221.287334] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2221.300644] alg: No test for adler32 (adler32-zlib) [ 2222.211308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2222.268540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2222.287880] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2223.065335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2223.137561] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping oleg117-client@tcp [ 2229.690581] Lustre: DEBUG MARKER: == sanity-lnet test 310: Set timeout and verify ========== 20:16:05 (1788653765) [ 2229.797130] LNet: 111761:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2229.806745] LNetError: 111761:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2229.832751] LNet: Removed LNI 192.168.201.17@tcp [ 2230.431500] Key type .llcrypt unregistered [ 2230.433311] Key type ._llcrypt unregistered [ 2236.698700] Key type ._llcrypt registered [ 2236.704856] Key type .llcrypt registered [ 2236.882195] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2236.907825] alg: No test for adler32 (adler32-zlib) [ 2237.876472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure --all [ 2237.931481] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2237.977896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp -v [ 2245.493361] Lustre: DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 20:16:20 (1788653780) [ 2247.356789] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type [ 2249.342483] Lustre: DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 20:16:24 (1788653784) [ 2250.822900] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type [ 2252.580957] Lustre: DEBUG MARKER: == sanity-lnet test 350: Check import --exec of ping/discover ========================================================== 20:16:27 (1788653787) [ 2253.100347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2253.124222] LNet: 113190:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2253.129768] LNetError: 113190:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2253.147955] LNet: Removed LNI 192.168.201.17@tcp [ 2253.165910] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2253.238731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2253.267341] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2253.987754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 2254.056761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 2261.968657] Lustre: DEBUG MARKER: == sanity-lnet test 400: Check for udsp add/delete net rule without net num ========================================================== 20:16:36 (1788653796) [ 2262.050771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2262.082699] LNet: 113893:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2262.097453] LNetError: 113893:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2262.117310] LNet: Removed LNI 192.168.201.17@tcp [ 2262.175610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2262.250791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --priority 0 [ 2262.277582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2262.915871] Key type .llcrypt unregistered [ 2262.917892] Key type ._llcrypt unregistered [ 2275.218148] Key type ._llcrypt registered [ 2275.219984] Key type .llcrypt registered [ 2276.809331] Lustre: DEBUG MARKER: == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 20:16:51 (1788653811) [ 2277.238954] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2277.251856] alg: No test for adler32 (adler32-zlib) [ 2278.195481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2278.248807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2278.265478] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2278.922213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2278.950329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2285.897478] Lustre: DEBUG MARKER: == sanity-lnet test 402: Destination net rule should not panic ========================================================== 20:17:00 (1788653820) [ 2286.014608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2286.038800] LNet: 115447:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2286.051077] LNetError: 115447:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2286.076671] LNet: Removed LNI 192.168.201.17@tcp [ 2286.130681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2286.210518] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst kfi --priority 0 [ 2286.241426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 402@kfi [ 2292.551513] Lustre: DEBUG MARKER: == sanity-lnet test 403: Add and delete udsp net and nid rules ========================================================== 20:17:07 (1788653827) [ 2292.763460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2292.787121] LNet: 116031:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2292.841086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2292.932081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2293.376309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2293.429643] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2293.526457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src 1.1.1.1@tcp --prio 1 [ 2294.138455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2294.163939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2294.217397] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2294.234326] LNet: 116065:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2294.252395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2294.308593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2294.968705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2295.029495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2295.151241] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst 1.1.1.1@tcp --prio 1 [ 2295.609281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2295.694670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2303.541337] Lustre: DEBUG MARKER: == sanity-lnet test 404: Check udsp src/dst net rule ===== 20:17:18 (1788653838) [ 2303.791992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2303.857452] LNet: 116666:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2303.897595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2303.971401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2304.036329] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2304.815336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2304.829766] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2305.751830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2305.833229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2306.133667] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2306.170836] LNetError: 116945:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2306.182738] LNet: Removed LNI 192.168.201.17@tcp [ 2306.192769] LNet: Removed LNI 192.168.201.17@tcp2 [ 2306.241267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2306.288263] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2306.294514] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2306.303535] LNet: 116951:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2306.860561] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2307.148724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2308.192017] LNet: Removed LNI 192.168.201.17@tcp [ 2308.196291] LNetError: 116989:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2308.204529] LNetError: 116989:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 2308.223297] LNet: Removed LNI 192.168.201.17@tcp2 [ 2308.242305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2308.312950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2308.335797] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2309.053668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2309.069701] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2309.900972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2309.937538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 2309.997555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2310.189116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2310.204126] LNet: 117273:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2310.208229] LNet: 117273:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 2310.212852] LNetError: 117273:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2310.218231] LNet: Removed LNI 192.168.201.17@tcp [ 2311.275773] LNet: Removed LNI 192.168.201.17@tcp2 [ 2311.317860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2311.385307] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2311.399983] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2311.403170] LNet: 117279:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2311.876862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2312.096628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2312.118203] LNet: Removed LNI 192.168.201.17@tcp [ 2313.213983] LNet: Removed LNI 192.168.201.17@tcp2 [ 2313.263297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2313.342272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2313.359736] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2314.138750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2314.157101] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2315.181245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2315.234696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2315.461180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2315.484581] LNetError: 117596:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2315.488870] LNetError: 117596:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 2316.513644] LNet: Removed LNI 192.168.201.17@tcp [ 2316.528215] LNet: Removed LNI 192.168.201.17@tcp2 [ 2316.576660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2316.649697] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2316.657594] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2316.663677] LNet: 117602:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2317.066597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2317.241624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2318.303212] LNet: Removed LNI 192.168.201.17@tcp [ 2318.313798] LNet: Removed LNI 192.168.201.17@tcp2 [ 2318.348504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2318.444785] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2318.500676] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2319.209673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2319.229030] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2320.018757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2320.052544] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 2320.115878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2320.318316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2320.333833] LNet: 117923:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2320.337611] LNet: 117923:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 2321.375289] LNet: Removed LNI 192.168.201.17@tcp [ 2322.406933] LNet: Removed LNI 192.168.201.17@tcp2 [ 2322.442569] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2322.521502] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2322.534833] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2322.541399] LNet: 117930:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2323.271583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2323.490889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2323.514896] LNetError: 117967:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2323.522757] LNetError: 117967:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 2323.528498] LNet: Removed LNI 192.168.201.17@tcp [ 2324.581702] LNet: Removed LNI 192.168.201.17@tcp2 [ 2324.606275] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2324.682952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2324.724849] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2325.319747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2325.331853] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2325.993562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2326.011080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2326.055743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2326.441163] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2327.455158] LNet: Removed LNI 192.168.201.17@tcp [ 2327.468918] LNet: Removed LNI 192.168.201.17@tcp2 [ 2327.525630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2327.627551] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2327.644231] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2327.650292] LNet: 118269:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2328.071285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2328.382514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2329.439206] LNet: Removed LNI 192.168.201.17@tcp [ 2329.453649] LNet: Removed LNI 192.168.201.17@tcp2 [ 2329.488581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2329.575875] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2329.596205] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2330.224780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2330.244858] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2331.165110] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2331.277771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 2331.340544] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 2331.455886] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2331.897730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2332.959228] LNet: Removed LNI 192.168.201.17@tcp [ 2333.989781] LNet: Removed LNI 192.168.201.17@tcp2 [ 2334.031617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2334.078191] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2334.081718] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2334.497382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2342.867240] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_410 skipping ALWAYS excluded test 410 [ 2344.425977] Lustre: DEBUG MARKER: == sanity-lnet test 450: Check import of multiple interfaces ========================================================== 20:17:59 (1788653879) [ 2344.626276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2344.641385] LNet: 119390:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2344.645566] LNet: 119390:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 4 previous similar messages [ 2344.650724] LNetError: 119390:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2344.654611] LNetError: 119390:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 7 previous similar messages [ 2344.668289] LNet: Removed LNI 192.168.201.17@tcp [ 2345.704566] LNet: Removed LNI 192.168.201.17@tcp2 [ 2345.756467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2346.136652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2346.180773] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2346.909472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2346.949497] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2347.896714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2347.918957] LNet: Removed LNI 10.1.2.3@tcp [ 2347.930945] LNet: Removed LNI 192.168.201.17@tcp [ 2347.953393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2348.041614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-450.yaml [ 2348.076636] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2348.080873] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2348.628181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del /tmp/sanity-lnet-450.yaml [ 2348.643773] LNet: Removed LNI 192.168.201.17@tcp [ 2348.656726] LNet: Removed LNI 10.1.2.3@tcp [ 2355.893153] Lustre: DEBUG MARKER: == sanity-lnet test 475: per-operation latency stats populate and reset ========================================================== 20:18:10 (1788653890) [ 2356.157562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2356.192484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2356.256562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2356.275896] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2356.971379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2356.997538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 2357.036434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2357.065824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2357.091366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2357.122937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2357.159192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2357.190832] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2357.233487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2357.276985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2357.321655] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2357.383281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2357.542911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 2364.456962] Lustre: DEBUG MARKER: == sanity-lnet test 476: lnetctl set latency_stats toggles capture ========================================================== 20:18:19 (1788653899) [ 2364.864792] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2365.928309] LNet: Removed LNI 192.168.201.17@tcp [ 2365.989180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2366.056765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2366.103346] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2366.750937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 0 [ 2366.783927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 2366.828940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2366.863349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2366.905879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2366.961735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2366.995095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.023216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.052611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.081758] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.132120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.169334] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.291722] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2367.374754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.446894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.531347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.568062] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.603495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.658965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.696849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.773479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.821843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2367.881230] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2375.853241] Lustre: DEBUG MARKER: == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 20:18:30 (1788653910) [ 2376.186527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2377.257748] LNet: Removed LNI 192.168.201.17@tcp [ 2377.331960] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2377.540818] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 2377.814435] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2377.821654] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2377.824107] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 2380.196638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2380.223691] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2381.015241] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2381.035960] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2383.242949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2383.305229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d *@tcp -r 1 -l 1 -m PUT [ 2383.398389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.201.17@tcp -m PUT -e local_timeout -r 1 [ 2383.513752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 10.1.2.3@tcp -m PUT -e local_timeout -r 1 [ 2383.599124] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2383.602822] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2383.826168] LNetError: 121828:0:(lib-move.c:4794:lnet_parse()) 192.168.201.17@tcp, src 192.168.201.17@tcp: Dropping PUT (error -108 looking up sender) [ 2384.863746] LNet: 122288:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2384.871294] LNet: 122288:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 4 previous similar messages [ 2384.890438] LNetError: 122288:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2384.907918] LNetError: 122288:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 2384.917518] LNet: Removed LNI 10.1.2.3@tcp [ 2385.957486] LNet: Removed LNI 192.168.201.17@tcp [ 2386.357934] Key type .llcrypt unregistered [ 2386.360162] Key type ._llcrypt unregistered [ 2400.770316] Key type ._llcrypt registered [ 2400.771865] Key type .llcrypt registered [ 2402.225547] Lustre: DEBUG MARKER: == sanity-lnet test 501: Verify lnetctl peer set --health (SR) ========================================================== 20:18:57 (1788653937) [ 2402.884808] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2402.900211] alg: No test for adler32 (adler32-zlib) [ 2403.914918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2404.375808] LNet: 123138:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2404.996488] Key type .llcrypt unregistered [ 2404.998173] Key type ._llcrypt unregistered [ 2411.666871] Key type ._llcrypt registered [ 2411.669365] Key type .llcrypt registered [ 2411.981034] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2412.007898] alg: No test for adler32 (adler32-zlib) [ 2413.005852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2413.048036] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2422.788836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 2423.107162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 2423.879404] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.117@tcp [ 2424.156849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2424.208906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2424.252506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2424.282409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2424.312794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2424.351266] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.117@tcp [ 2424.352504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2424.388643] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2424.426618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2424.464877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2424.514333] LNet: 123596:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2424.554257] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2424.606701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2424.645107] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2424.674654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2424.704658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2424.734037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2425.374331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 2425.663604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2425.723916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2425.749933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2425.770933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2425.798135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2425.824829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2425.863948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2425.906950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2425.948964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2425.972974] LNet: 123715:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2425.996252] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2426.056121] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2426.093381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2426.127407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2426.190437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2426.240298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2426.978120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.117@tcp [ 2427.322671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2427.380179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2427.437294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2427.512996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2427.584794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2427.631565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2427.665239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2427.690941] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2427.721338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2427.741515] LNet: 123834:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2427.760791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2427.814552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2427.836783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2427.857187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2427.903907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2427.945717] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2428.822585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 2429.071302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2429.103898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2429.134079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2429.185555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2429.254807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2429.318693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2429.358838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2429.418796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2429.477824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2429.537955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2429.586725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2429.630954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2429.664610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2429.718681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2429.764842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2430.143327] LNet: 124000:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2430.149053] LNetError: 124000:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2430.171610] LNet: Removed LNI 192.168.201.17@tcp [ 2430.637025] Key type .llcrypt unregistered [ 2430.639484] Key type ._llcrypt unregistered [ 2450.132612] Key type ._llcrypt registered [ 2450.137052] Key type .llcrypt registered [ 2451.804898] Lustre: DEBUG MARKER: == sanity-lnet test 502: Verify lnetctl peer set --health (MR) ========================================================== 20:19:46 (1788653986) [ 2452.211547] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2452.230870] alg: No test for adler32 (adler32-zlib) [ 2453.230898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2453.905916] LNet: 124884:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2454.701498] Key type .llcrypt unregistered [ 2454.708602] Key type ._llcrypt unregistered [ 2461.726538] Key type ._llcrypt registered [ 2461.728729] Key type .llcrypt registered [ 2461.921447] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2461.951900] alg: No test for adler32 (adler32-zlib) [ 2463.056574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2463.098599] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2471.855693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 2480.789781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2480.823899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2480.850224] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 2481.761378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 2482.275310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.117@tcp [ 2482.466508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2482.486191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2482.512652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2482.536482] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2482.563450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2482.594888] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.117@tcp [ 2482.621474] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2482.662378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2482.709139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2482.740324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2482.765712] LNet: 125535:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2482.780760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2482.799666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2482.826786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2482.854251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2482.885638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2482.924216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2483.738454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 2483.989771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2484.018069] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2484.040471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2484.065115] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2484.096307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2484.124629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2484.151870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2484.193747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2484.237925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2484.259106] LNet: 125654:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2484.297856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2484.334880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2484.392372] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2484.456200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2484.502989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2484.546271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2485.363275] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.117@tcp [ 2485.567505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2485.606810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2485.648641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2485.693905] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2485.726917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2485.772847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2485.802460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2485.835458] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2485.860878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2485.884987] LNet: 125773:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2485.907107] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2485.965317] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2486.012834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2486.051733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2486.099470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2486.145777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2486.876435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 2487.216426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2487.246347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2487.290849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2487.326862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2487.368728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2487.396830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2487.424344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2487.460664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2487.482912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2487.512959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2487.557599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2487.609650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2487.665179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2487.695485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2487.720479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2493.146407] LNet: 125973:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2493.155788] LNetError: 125973:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2494.175316] LNet: Removed LNI 192.168.201.17@tcp [ 2494.179155] LNetError: 125973:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2494.199707] LNet: Removed LNI 192.168.201.17@tcp1 [ 2494.821651] Key type .llcrypt unregistered [ 2494.823282] Key type ._llcrypt unregistered [ 2512.934705] Key type ._llcrypt registered [ 2512.937792] Key type .llcrypt registered [ 2514.634154] Lustre: DEBUG MARKER: == sanity-lnet test 525: keep reachable peer on local net delete ========================================================== 20:20:49 (1788654049) [ 2515.129318] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2515.142742] alg: No test for adler32 (adler32-zlib) [ 2516.132052] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2516.195419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2516.226525] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2516.977758] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2517.007463] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2518.287302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2518.388628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp [ 2519.456335] LNet: Removed LNI 192.168.201.17@tcp [ 2526.619730] Lustre: DEBUG MARKER: == sanity-lnet test 550: DD-off: keep route up when gw reachable via another local NI ========================================================== 20:21:02 (1788654062) [ 2528.668468] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_550 Need at least 2 local interfaces for multi-rail [ 2530.787854] Lustre: DEBUG MARKER: == sanity-lnet test 600: lnet_legacy2yaml: basic ip2nets conversion ========================================================== 20:21:06 (1788654066) [ 2538.050348] Lustre: DEBUG MARKER: == sanity-lnet test 601: lnet_legacy2yaml: basic routes conversion ========================================================== 20:21:13 (1788654073) [ 2545.996227] Lustre: DEBUG MARKER: == sanity-lnet test 602: lnet_legacy2yaml: routes with priority ========================================================== 20:21:21 (1788654081) [ 2554.125480] Lustre: DEBUG MARKER: == sanity-lnet test 603: lnet_legacy2yaml: default hop flag ========================================================== 20:21:28 (1788654088) [ 2563.679754] Lustre: DEBUG MARKER: == sanity-lnet test 604: lnet_legacy2yaml: default hop with explicit value ========================================================== 20:21:38 (1788654098) [ 2573.943417] Lustre: DEBUG MARKER: == sanity-lnet test 605: lnet_legacy2yaml: quote leading star in IP pattern ========================================================== 20:21:48 (1788654108) [ 2582.072631] Lustre: DEBUG MARKER: == sanity-lnet test 606: lnet_legacy2yaml: file inputs === 20:21:57 (1788654117) [ 2590.749772] Lustre: DEBUG MARKER: == sanity-lnet test 607: lnet_legacy2yaml: options line with combined ip2nets and routes ========================================================== 20:22:05 (1788654125) [ 2599.922318] Lustre: DEBUG MARKER: == sanity-lnet test 608: lnet_legacy2yaml: error on invalid rule (missing IP pattern) ========================================================== 20:22:14 (1788654134) [ 2607.506084] Lustre: DEBUG MARKER: == sanity-lnet test 609: lnet_legacy2yaml: error on no input ========================================================== 20:22:22 (1788654142) [ 2613.434274] Lustre: DEBUG MARKER: == sanity-lnet test 610: lnet_legacy2yaml: multiple files and inline, check order ========================================================== 20:22:28 (1788654148) [ 2620.762838] Lustre: DEBUG MARKER: == sanity-lnet test 611: lnet_legacy2yaml: invalid hopcount ========================================================== 20:22:35 (1788654155) [ 2627.717488] Lustre: DEBUG MARKER: == sanity-lnet test 612: lnet_legacy2yaml: malformed IP pattern ========================================================== 20:22:43 (1788654163) [ 2634.785147] Lustre: DEBUG MARKER: == sanity-lnet test 613: lnet_legacy2yaml: multiple IP patterns in single rule ========================================================== 20:22:49 (1788654169) [ 2642.630663] Lustre: DEBUG MARKER: == sanity-lnet test 614: lnet_legacy2yaml: interfaces enumeration with multiple IP ranges ========================================================== 20:22:57 (1788654177) [ 2650.198712] Lustre: DEBUG MARKER: == sanity-lnet test 615: lnet_legacy2yaml: modprobe configuration file input ========================================================== 20:23:04 (1788654184) [ 2660.086462] Lustre: DEBUG MARKER: == sanity-lnet test 616: lnet_legacy2yaml: combining modprobe file with explicit ip2nets file ========================================================== 20:23:14 (1788654194) [ 2669.421965] Lustre: DEBUG MARKER: == sanity-lnet test 617: lnet_legacy2yaml: round-trip tunables from LNet to modprobe to YAML ========================================================== 20:23:24 (1788654204) [ 2669.670526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2669.729758] LNet: 137666:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2669.747472] LNetError: 137666:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2669.774462] LNet: Removed LNI 192.168.201.17@tcp2 [ 2669.797452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2669.990727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2670.044627] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2671.397736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2671.416754] LNet: 137809:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2671.426758] LNetError: 137809:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2671.435761] LNet: Removed LNI 192.168.201.17@tcp [ 2671.449850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2671.502340] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 2671.565555] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2679.771504] Lustre: DEBUG MARKER: == sanity-lnet test 618: lnet_legacy2yaml: alternative route format ========================================================== 20:23:34 (1788654214) [ 2687.472639] Lustre: DEBUG MARKER: == sanity-lnet test 619: lnet_legacy2yaml: multiple alternative format routes from file ========================================================== 20:23:42 (1788654222) [ 2695.278207] Lustre: DEBUG MARKER: == sanity-lnet test 620: lnet_legacy2yaml: mixed legacy and alternative route formats ========================================================== 20:23:50 (1788654230) [ 2703.146150] Lustre: DEBUG MARKER: == sanity-lnet test 621: lnet_legacy2yaml: alternative format with invalid hopcount ========================================================== 20:23:58 (1788654238) [ 2709.706797] Lustre: DEBUG MARKER: == sanity-lnet test 622: lnet_legacy2yaml: alternative format with default-hop flag ========================================================== 20:24:04 (1788654244) [ 2716.210744] Lustre: DEBUG MARKER: == sanity-lnet test 623: lnet_legacy2yaml: basic networks conversion ========================================================== 20:24:11 (1788654251) [ 2723.160503] Lustre: DEBUG MARKER: == sanity-lnet test 624: lnet_legacy2yaml: networks with interfaces ========================================================== 20:24:18 (1788654258) [ 2730.040235] Lustre: DEBUG MARKER: == sanity-lnet test 625: lnet_legacy2yaml: multiple networks ========================================================== 20:24:25 (1788654265) [ 2737.561153] Lustre: DEBUG MARKER: == sanity-lnet test 626: lnet_legacy2yaml: mixed networks, ip2nets, and routes ========================================================== 20:24:32 (1788654272) [ 2744.321606] Lustre: DEBUG MARKER: == sanity-lnet test 627: lnet_legacy2yaml: invalid networks format ========================================================== 20:24:39 (1788654279) [ 2750.974883] Lustre: DEBUG MARKER: == sanity-lnet test 628: lnet_legacy2yaml: missing delimiter error ========================================================== 20:24:46 (1788654286) [ 2757.470121] Lustre: DEBUG MARKER: == sanity-lnet test 629: lnet_legacy2yaml: multiple gateways error ========================================================== 20:24:52 (1788654292) [ 2764.089376] Lustre: DEBUG MARKER: == sanity-lnet test complete, duration 2504 sec ========== 20:24:58 (1788654298) [ 2764.448276] LNet: 145272:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2764.461294] LNetError: 145272:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2764.485127] LNet: Removed LNI 192.168.201.17@tcp [ 2765.262816] Key type .llcrypt unregistered [ 2765.268129] Key type ._llcrypt unregistered [ 2777.192432] Key type ._llcrypt registered [ 2777.194208] Key type .llcrypt registered [ 2777.536107] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2777.547188] alg: No test for adler32 (adler32-zlib) [ 2778.739812] Lustre: Lustre: Build Version: 2.17.58_2_g557ce73 [ 2779.033953] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2780.719545] Key type lgssc registered [ 2781.977526] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2849.345956] Lustre: Mounted lustre-client - version 2.17.58_2_g557ce73 [ 2854.996872] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2863.663055] Lustre: Unmounted lustre-client [ 2888.736520] Key type lgssc unregistered [ 2889.049540] LNet: 148223:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2889.066856] LNetError: 148223:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2890.093254] LNet: Removed LNI 192.168.201.17@tcp [ 2890.991186] Key type .llcrypt unregistered [ 2890.994790] Key type ._llcrypt unregistered