[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 484950892 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54d0-0x000f54df] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829700K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002271] x2apic enabled [ 0.003016] Switched APIC routing to physical x2apic. [ 0.004011] kvm-guest: setup PV IPIs [ 0.007464] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008030] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010006] pid_max: default: 32768 minimum: 301 [ 0.011119] LSM: Security Framework initializing [ 0.012044] Yama: becoming mindful. [ 0.013028] SELinux: Initializing. [ 0.014064] *** VALIDATE selinux *** [ 0.021740] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.025646] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027106] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028082] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029112] *** VALIDATE tmpfs *** [ 0.030427] *** VALIDATE proc *** [ 0.032181] *** VALIDATE cgroup *** [ 0.033008] *** VALIDATE cgroup2 *** [ 0.035061] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036126] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038028] Spectre V2 : User space: Vulnerable [ 0.039005] Speculative Store Bypass: Vulnerable [ 0.042388] debug: unmapping init [mem 0xffffffffa6a59000-0xffffffffa6a60fff] [ 0.044823] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045663] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046020] ... version: 2 [ 0.047012] ... bit width: 48 [ 0.048012] ... generic registers: 4 [ 0.049010] ... value mask: 0000ffffffffffff [ 0.050010] ... max period: 00007fffffffffff [ 0.051009] ... fixed-purpose events: 3 [ 0.052007] ... event mask: 000000070000000f [ 0.053264] rcu: Hierarchical SRCU implementation. [ 0.055326] smp: Bringing up secondary CPUs ... [ 0.056477] x86: Booting SMP configuration: [ 0.057020] .... node #0, CPUs: #1 #2 #3 [ 0.060230] smp: Brought up 1 node, 4 CPUs [ 0.062013] smpboot: Max logical packages: 1 [ 0.063013] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.138511] node 0 deferred pages initialised in 72ms [ 0.142111] devtmpfs: initialized [ 0.143254] x86/mm: Memory block size: 128MB [ 0.146825] gcov: version magic: 0x41383552 [ 0.147579] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.150071] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.152309] pinctrl core: initialized pinctrl subsystem [ 0.153128] [ 0.153495] ************************************************************* [ 0.155008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.157008] ** ** [ 0.158006] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.160008] ** ** [ 0.161008] ** This means that this kernel is built to expose internal ** [ 0.163007] ** IOMMU data structures, which may compromise security on ** [ 0.164007] ** your system. ** [ 0.166007] ** ** [ 0.168008] ** If you see this message and you are not debugging the ** [ 0.169008] ** kernel, report this immediately to your vendor! ** [ 0.171007] ** ** [ 0.173008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.174007] ************************************************************* [ 0.176673] NET: Registered protocol family 16 [ 0.178403] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.180039] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.181089] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.184079] cpuidle: using governor menu [ 0.186000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.187459] PCI: Using configuration type 1 for base access [ 0.189188] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.196278] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.197000] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.199025] cryptd: max_cpu_qlen set to 1000 [ 0.200216] ACPI: Added _OSI(Module Device) [ 0.201000] ACPI: Added _OSI(Processor Device) [ 0.201000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.204012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.209381] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.214403] ACPI: Interpreter enabled [ 0.216060] ACPI: PM: (supports S0 S3 S4 S5) [ 0.218011] ACPI: Using IOAPIC for interrupt routing [ 0.219086] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.222471] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.234887] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.237033] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.240014] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.243071] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.247473] acpiphp: Slot [2] registered [ 0.248125] acpiphp: Slot [5] registered [ 0.249070] acpiphp: Slot [6] registered [ 0.250062] acpiphp: Slot [3] registered [ 0.251093] acpiphp: Slot [4] registered [ 0.253129] acpiphp: Slot [7] registered [ 0.254069] acpiphp: Slot [8] registered [ 0.256079] acpiphp: Slot [9] registered [ 0.258172] acpiphp: Slot [10] registered [ 0.259084] acpiphp: Slot [11] registered [ 0.261078] acpiphp: Slot [12] registered [ 0.263078] acpiphp: Slot [13] registered [ 0.264079] acpiphp: Slot [14] registered [ 0.266073] acpiphp: Slot [15] registered [ 0.268081] acpiphp: Slot [16] registered [ 0.269175] acpiphp: Slot [17] registered [ 0.271072] acpiphp: Slot [18] registered [ 0.273076] acpiphp: Slot [19] registered [ 0.274192] acpiphp: Slot [20] registered [ 0.276078] acpiphp: Slot [21] registered [ 0.277083] acpiphp: Slot [22] registered [ 0.279111] acpiphp: Slot [23] registered [ 0.280095] acpiphp: Slot [24] registered [ 0.282101] acpiphp: Slot [25] registered [ 0.284127] acpiphp: Slot [26] registered [ 0.285078] acpiphp: Slot [27] registered [ 0.287074] acpiphp: Slot [28] registered [ 0.289079] acpiphp: Slot [29] registered [ 0.291131] acpiphp: Slot [30] registered [ 0.292096] acpiphp: Slot [31] registered [ 0.294066] PCI host bridge to bus 0000:00 [ 0.296078] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.298022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.301018] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.304021] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.306019] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.309053] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.311126] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.313787] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.315879] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.321943] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.325600] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.328013] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.330013] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.332013] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.334387] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.336550] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.338020] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.339452] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.343011] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.351016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.355010] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.359911] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.371014] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.379012] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.389014] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.401512] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.419018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.433014] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.484016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.502010] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.503310] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.505224] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.506302] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.508141] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.511093] iommu: Default domain type: Passthrough [ 0.513424] SCSI subsystem initialized [ 0.515118] ACPI: bus type USB registered [ 0.516177] usbcore: registered new interface driver usbfs [ 0.518071] usbcore: registered new interface driver hub [ 0.520057] usbcore: registered new device driver usb [ 0.521099] pps_core: LinuxPPS API ver. 1 registered [ 0.523009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.527051] PTP clock support registered [ 0.530052] EDAC MC: Ver: 3.0.0 [ 0.532136] PCI: Using ACPI for IRQ routing [ 0.533660] NetLabel: Initializing [ 0.535009] NetLabel: domain hash size = 128 [ 0.536007] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.538091] NetLabel: unlabeled traffic allowed by default [ 0.540106] vgaarb: loaded [ 0.541215] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.543010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.551143] clocksource: Switched to clocksource kvm-clock [ 0.645051] VFS: Disk quotas dquot_6.6.0 [ 0.646018] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.647682] *** VALIDATE ramfs *** [ 0.648480] *** VALIDATE hugetlbfs *** [ 0.649535] pnp: PnP ACPI init [ 0.651267] pnp: PnP ACPI: found 6 devices [ 0.688606] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.690851] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.692256] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.693634] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.695346] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.696774] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.698584] NET: Registered protocol family 2 [ 0.700385] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.704235] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.706500] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.710424] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.712670] TCP: Hash tables configured (established 65536 bind 65536) [ 0.714653] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.716704] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.718553] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.720549] NET: Registered protocol family 1 [ 0.723687] RPC: Registered named UNIX socket transport module. [ 0.725025] RPC: Registered udp transport module. [ 0.726036] RPC: Registered tcp transport module. [ 0.727222] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.728699] NET: Registered protocol family 44 [ 0.730379] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.731799] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.733126] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.734597] PCI: CLS 0 bytes, default 64 [ 0.736305] Unpacking initramfs... [ 2.166934] debug: unmapping init [mem 0xffff9d937cc64000-0xffff9d937ffcffff] [ 2.170648] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.172784] software IO TLB: mapped [mem 0x00000000a1800000-0x00000000a5800000] (64MB) [ 2.175106] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.693096] Initialise system trusted keyrings [ 2.694482] Key type blacklist registered [ 2.697063] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.704678] zbud: loaded [ 2.707399] *** VALIDATE nfs *** [ 2.708731] *** VALIDATE nfs4 *** [ 2.710137] pstore: using deflate compression [ 2.713185] Platform Keyring initialized [ 2.814961] NET: Registered protocol family 38 [ 2.817159] Key type asymmetric registered [ 2.818471] Asymmetric key parser 'x509' registered [ 2.819822] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.822626] io scheduler mq-deadline registered [ 2.823961] io scheduler kyber registered [ 2.825989] io scheduler bfq registered [ 2.827587] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.830374] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.833507] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.835814] ACPI: Power Button [PWRF] [ 2.842209] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.848592] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.866918] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.893769] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.920703] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.924828] Non-volatile memory driver v1.3 [ 2.926684] Linux agpgart interface v0.103 [ 2.955637] virtio_blk virtio1: [vda] 145904 512-byte logical blocks (74.7 MB/71.2 MiB) [ 2.958885] vda: detected capacity change from 0 to 74702848 [ 2.994675] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.997818] vdb: detected capacity change from 0 to 1073741824 [ 3.019892] libphy: Fixed MDIO Bus: probed [ 3.033042] usbcore: registered new interface driver usbserial_generic [ 3.036957] usbserial: USB Serial support registered for generic [ 3.039356] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.043495] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.045380] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.047714] mousedev: PS/2 mouse device common for all mice [ 3.050441] rtc_cmos 00:05: RTC can wake from S4 [ 3.053760] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.057922] rtc_cmos 00:05: registered as rtc0 [ 3.059993] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.060638] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.062471] intel_pstate: CPU model not supported [ 3.067530] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.072352] hid: raw HID events driver (C) Jiri Kosina [ 3.074076] usbcore: registered new interface driver usbhid [ 3.075610] usbhid: USB HID core driver [ 3.076733] drop_monitor: Initializing network drop monitor service [ 3.079666] Initializing XFRM netlink socket [ 3.081119] NET: Registered protocol family 10 [ 3.083375] Segment Routing with IPv6 [ 3.084548] NET: Registered protocol family 17 [ 3.086027] mpls_gso: MPLS GSO support [ 3.092573] RAS: Correctable Errors collector initialized. [ 3.094382] AVX version of gcm_enc/dec engaged. [ 3.095554] AES CTR mode by8 optimization enabled [ 3.186757] sched_clock: Marking stable (3186734547, 0)->(3969182860, -782448313) [ 3.190167] registered taskstats version 1 [ 3.193158] Loading compiled-in X.509 certificates [ 3.195831] zswap: loaded using pool lzo/zbud [ 3.225080] Key type big_key registered [ 3.237810] Key type encrypted registered [ 3.240119] ima: No TPM chip found, activating TPM-bypass! [ 3.242828] ima: Allocated hash algorithm: sha1 [ 3.245089] ima: No architecture policies found [ 3.247097] evm: Initialising EVM extended attributes: [ 3.249445] evm: security.selinux [ 3.251210] evm: security.ima [ 3.252740] evm: security.capability [ 3.254500] evm: HMAC attrs: 0x1 [ 3.258262] rtc_cmos 00:05: setting system clock to 2026-08-04 20:44:48 UTC (1785876288) [ 3.267364] debug: unmapping init [mem 0xffffffffa7a03000-0xffffffffa7bfffff] [ 3.270220] debug: unmapping init [mem 0xffffffffa6782000-0xffffffffa6a58fff] [ 3.284314] Write protecting the kernel read-only data: 28672k [ 3.289455] debug: unmapping init [mem 0xffffffffa4e03000-0xffffffffa4ffffff] [ 3.293526] debug: unmapping init [mem 0xffffffffa5714000-0xffffffffa57fffff] [ 3.324722] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.336709] systemd[1]: Detected virtualization kvm. [ 3.338803] systemd[1]: Detected architecture x86-64. [ 3.340763] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.367798] systemd[1]: No hostname configured. [ 3.369536] systemd[1]: Set hostname to . [ 3.371819] random: systemd: uninitialized urandom read (16 bytes read) [ 3.374056] systemd[1]: Initializing machine ID from random generator. [ 3.420808] random: ln: uninitialized urandom read (6 bytes read) [ 3.492407] random: systemd: uninitialized urandom read (16 bytes read) [ 3.494141] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.497296] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.502098] systemd[1]: Reached target Local Encrypted Volumes. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Timers. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket. [ OK ] Reached target Sockets. Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. Starting Journal Service... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook...[ 4.043720] random: fast init done [ 4.125424] device-mapper: uevent: version 1.0.3 [ 4.127915] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.920059] virtio_net virtio0 ens2: renamed from eth0 [ 4.985429] scsi host0: ata_piix [ 4.989535] scsi host1: ata_piix [ 5.049823] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 5.052805] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 9.331890] dracut-initqueue[589]: RTNETLINK answers: File exists [ 9.763681] random: crng init done [ 9.764663] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.165250] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.350823] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.685870] SELinux: Disabled at runtime. [ 11.757897] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.768502] systemd[1]: Detected virtualization kvm. [ 11.770464] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.451793] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.455262] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.460037] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.465083] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.468415] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.476373] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.488092] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target rpc_pipefs.target. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice system-serial\x2dgetty.slice. Starting Apply Kernel Variables... Mounting Huge Pages File System... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on udev Control Socket. Mounting POSIX Message Queue File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Initrd Root File System. Mounting Kernel Debug File System... [ OK ] Listening on Process Core Dump Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice User and Session Slice. Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [ OK ] Reached target RPC Port Mapper. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-getty.slice. Starting Remount Root and Kernel File Systems... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] [ 12.651604] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Reached target Paths. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.993361] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.299295] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.362179] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.452935] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.464923] EDAC sbridge: Ver: 1.1.2 [ 14.594827] Key type dns_resolver registered [ 14.914214] NFS: Registering the id_resolver key type [ 14.916043] Key type id_resolver registered [ 14.917440] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started D-Bus System Message Bus. [ OK ] Started irqbalance daemon. Starting Network Manager... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target sshd-keygen.target. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg105-client login: [ 43.795291] libcfs: loading out-of-tree module taints kernel. [ 43.833629] Key type ._llcrypt registered [ 43.834920] Key type .llcrypt registered [ 44.047892] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 44.055454] alg: No test for adler32 (adler32-zlib) [ 45.102370] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 45.455285] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 47.079204] Key type lgssc registered [ 47.770294] Lustre: Echo OBD driver; http://www.lustre.org/ [ 227.084107] Lustre: Mounted lustre-client - version 2.17.56_50_g55fd350 [ 235.966461] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 252.896261] Lustre: lustre-OST0000-osc-ffff9d93ca22d000: disconnect after 22s idle [ 253.799382] Lustre: DEBUG MARKER: oleg105-client.virtnet: executing check_logdir /tmp/testlogs/ [ 258.578603] hrtimer: interrupt took 5594904 ns [ 263.266917] Lustre: DEBUG MARKER: oleg105-client.virtnet: executing yml_node [ 268.927788] Lustre: DEBUG MARKER: Client: 2.17.56.50 [ 272.226988] Lustre: DEBUG MARKER: MDS: 2.17.56.50 [ 276.214444] Lustre: DEBUG MARKER: OSS: 2.17.56.50 [ 278.985955] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Tue Aug 4 16:49:22 EDT 2026 [ 300.723657] Lustre: Unmounted lustre-client [ 342.031666] Key type lgssc unregistered [ 342.448046] LNet: 8418:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 342.456348] LNetError: 8418:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 342.497099] LNet: Removed LNI 192.168.201.5@tcp [ 343.732191] Key type .llcrypt unregistered [ 343.733660] Key type ._llcrypt unregistered [ 380.275082] Key type ._llcrypt registered [ 380.276557] Key type .llcrypt registered [ 380.563768] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 380.602765] alg: No test for adler32 (adler32-zlib) [ 381.556209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 381.647889] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 381.692961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show [ 386.551982] Lustre: DEBUG MARKER: excepting tests: 253 254 410 [ 386.816452] LNet: 9308:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 386.823854] LNetError: 9308:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 386.851086] LNet: Removed LNI 192.168.201.5@tcp [ 387.514483] Key type .llcrypt unregistered [ 387.517449] Key type ._llcrypt unregistered [ 395.989840] Key type ._llcrypt registered [ 395.990922] Key type .llcrypt registered [ 399.156836] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 16:51:20 (1785876680) [ 399.967607] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 400.045569] alg: No test for adler32 (adler32-zlib) [ 401.360653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 401.517810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 401.558300] LNet: 9723:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 401.593280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 401.713953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 401.752949] LNet: 9730:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 412.450137] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 16:51:35 (1785876695) [ 412.553722] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 412.587246] LNet: 10300:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 412.630916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 412.743716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp [ 421.066629] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 16:51:44 (1785876704) [ 421.433120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 421.470226] LNet: 10896:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 421.505835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 421.603622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 2.2.2.2@o2ib [ 430.203793] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 16:51:53 (1785876713) [ 431.047288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 431.084568] LNet: 11481:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 431.119963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 431.238174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib [ 441.604896] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 16:52:04 (1785876724) [ 441.717081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 441.748814] LNet: 12061:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 441.773927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 441.896336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp [ 442.448624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp [ 450.853966] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 16:52:14 (1785876734) [ 450.971323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 450.996866] LNet: 12655:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 451.030289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 451.127290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 451.697723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 452.209981] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib [ 459.125170] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 16:52:22 (1785876742) [ 459.401874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 459.504236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 459.571806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi,[6-12/4]@bxi3f [ 467.953612] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 16:52:31 (1785876751) [ 468.458402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 468.507386] LNet: 13842:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 468.515339] LNet: 13842:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 468.566295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 468.661926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp [ 468.697762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 469.275447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib [ 469.310360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 469.981740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp [ 470.029658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 470.628268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib [ 470.667990] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 471.289609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib [ 471.341787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 472.047465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni [ 472.108857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 472.885093] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni [ 472.946444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 473.869670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi [ 473.939944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 474.878290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi [ 474.948376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 475.608809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f [ 475.686929] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 476.226276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f --nid [8-12]@bxi3f [ 476.272379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 476.714088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi,[3-7]@bxi3f [ 476.772570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 477.415698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 477.458898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 488.263613] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 16:52:51 (1785876771) [ 488.773556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 488.832502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 488.999237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp [ 489.101833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp [ 499.304927] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 16:53:02 (1785876782) [ 499.486785] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 499.592426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 499.743902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 499.859980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 508.933400] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 16:53:13 (1785876793) [ 509.373810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 509.407729] LNet: 15701:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 509.411432] LNet: 15701:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 509.444775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 509.544260] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp [ 509.616974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp [ 517.960825] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 16:53:21 (1785876801) [ 518.246859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 518.366532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 518.494186] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 518.573961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 527.326629] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 16:53:31 (1785876811) [ 528.396470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 528.473698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 528.603587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp [ 528.674405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp [ 538.743280] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 16:53:42 (1785876822) [ 538.965381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 539.067823] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 539.260276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 539.360540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 547.503923] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 16:53:51 (1785876831) [ 548.008207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 548.053440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 548.167300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 548.654635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 557.346711] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 16:54:01 (1785876841) [ 557.851461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 557.967784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 558.223710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 558.548893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 567.962551] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 16:54:11 (1785876851) [ 568.037489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 568.071585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 568.206449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 568.502630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 579.890947] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 16:54:23 (1785876863) [ 580.015150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 580.051048] LNet: 19871:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 580.066381] LNet: 19871:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 6 previous similar messages [ 580.094537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 580.866677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 581.254391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 592.914459] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 16:54:35 (1785876875) [ 593.235915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 593.285147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 593.401538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 593.510848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 603.505975] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 16:54:46 (1785876886) [ 603.671300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 603.732556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 603.909930] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 603.986273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 612.770672] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 16:54:56 (1785876896) [ 613.232951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 613.307029] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 613.436337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 19@gni [ 621.318637] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 16:55:05 (1785876905) [ 621.833898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 621.905653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 622.025791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib [ 630.557031] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 16:55:13 (1785876913) [ 630.715027] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 630.845414] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 630.995873] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni [ 631.831754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni [ 640.234600] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 16:55:23 (1785876923) [ 640.671750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 640.763744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 640.888234] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni [ 640.921654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 22@gni --nid 26@gni [ 650.217140] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 16:55:33 (1785876933) [ 650.454435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 650.593554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 650.744172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni [ 650.801491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni [ 660.395019] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 16:55:43 (1785876943) [ 660.832140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 660.908626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 661.067675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni [ 661.121505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp [ 670.150376] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 16:55:53 (1785876953) [ 670.419046] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 670.474355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 670.627986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 670.684204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 680.403744] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 16:56:03 (1785876963) [ 680.584958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 680.691730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 680.804250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim [ 680.883835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp [ 680.902512] LNetError: 25854:0:(peer.c:2362:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 680.935689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --force [ 680.975848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show --nid 1.1.1.1@tcp [ 688.700133] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 16:56:12 (1785876972) [ 688.809599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 688.865531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 688.988367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.FfaFa1jEql [ 689.055601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.FfaFa1jEql [ 689.115327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.FfaFa1jEql [ 697.889097] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 16:56:20 (1785876980) [ 698.038559] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 698.114244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 698.248311] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp [ 698.312210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp [ 698.373632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer list [ 705.592123] Lustre: DEBUG MARKER: == sanity-lnet test 40: LNetAddPeer merges a pre-existing non-primary peer NID ========================================================== 16:56:29 (1785876989) [ 705.815566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 705.898749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 706.008245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 706.041243] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 707.956105] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 709.754403] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 712.039643] Key type lgssc registered [ 713.492841] Lustre: Echo OBD driver; http://www.lustre.org/ [ 717.205897] LNetError: 27616:0:(lib-move.c:2158:lnet_handle_find_routed_path()) peer 192.168.201.5@tcp99 has no available nets [ 719.949568] Key type lgssc unregistered [ 720.294385] LNet: 28727:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 720.299930] LNet: 28727:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 13 previous similar messages [ 720.318843] LNetError: 28727:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 720.338372] LNet: Removed LNI 192.168.201.5@tcp [ 721.042709] Key type .llcrypt unregistered [ 721.048572] Key type ._llcrypt unregistered [ 728.571720] Key type ._llcrypt registered [ 728.576411] Key type .llcrypt registered [ 730.187096] Lustre: DEBUG MARKER: == sanity-lnet test 50: Enable/disable routing via yaml import ========================================================== 16:56:53 (1785877013) [ 730.698979] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 730.723184] alg: No test for adler32 (adler32-zlib) [ 731.732425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 731.807678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 735.365388] LNet: Message forwarding will be enabled in 2s [ 738.466198] LNet: Message forwarding enabled [ 738.987320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 739.016680] LNet: Message forwarding will stop in 2s to 6s [ 743.756542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 745.781943] LNet: Message forwarding disabled [ 748.553528] LNet: Message forwarding will be enabled in 2s [ 751.967113] LNet: Message forwarding enabled [ 757.216853] Lustre: DEBUG MARKER: == sanity-lnet test 51: Set numa range via yaml import === 16:57:20 (1785877040) [ 757.591405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 757.639613] LNet: 30139:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 760.020868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 760.139970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-51.yaml [ 768.368980] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 16:57:31 (1785877051) [ 768.488898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 768.529925] LNet: 30715:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 768.574251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 768.662972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid foobar [ 768.712219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid foobar [ 768.747879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@o2ib [ 768.795269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --nid 1.1.1.1@tcp [ 768.844228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --nid 1.1.1.1@tcp [ 768.901742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add [ 768.958541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 769.019880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp [ 769.096859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@foo [ 769.140952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@tcp [ 769.189581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@o2ib [ 769.228895] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@gni [ 769.269992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni [ 769.307898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni [ 769.350975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp [ 769.394037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp [ 769.446970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp [ 769.509966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 769.559901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp --gateway 1@gni [ 769.593421] LNetError: 30765:0:(router.c:625:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni [ 777.988691] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 16:57:41 (1785877061) [ 778.419501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 778.445854] LNet: 31336:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 778.469241] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 778.519666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 785.840642] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 16:57:49 (1785877069) [ 787.922915] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_100 Need at least 2 remote nodes found oleg105-server [ 790.157716] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 16:57:53 (1785877073) [ 790.548390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 790.592563] LNet: 32268:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 790.641873] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 790.733124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 790.783150] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 792.011870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp1 --gateway 192.168.201.[6-10]@tcp [ 792.047632] LNet: 32402:0:(router.c:729:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 792.123866] LNetError: 32272:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.7@tcp is being used as a gateway but routing feature is not turned on [ 792.136992] LNetError: 32272:0:(lib-lnet.h:1274:lnet_set_route_aliveness()) route to tcp1 through 192.168.201.7@tcp has gone from up to down [ 793.139399] LNetError: 32272:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.6@tcp is being used as a gateway but routing feature is not turned on [ 793.155323] LNetError: 32272:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 9 previous similar messages [ 794.156488] LNetError: 32272:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.9@tcp is being used as a gateway but routing feature is not turned on [ 794.174527] LNetError: 32272:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 4 previous similar messages [ 796.223727] LNetError: 32272:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.6@tcp is being used as a gateway but routing feature is not turned on [ 796.245564] LNetError: 32272:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 9 previous similar messages [ 800.291366] LNetError: 32272:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.9@tcp is being used as a gateway but routing feature is not turned on [ 800.314922] LNetError: 32272:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 19 previous similar messages [ 802.926336] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 16:58:05 (1785877085) [ 806.566363] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_102 Need at least 2 remote nodes found oleg105-server [ 808.498377] LNetError: 32272:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.7@tcp is being used as a gateway but routing feature is not turned on [ 808.510738] LNetError: 32272:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 39 previous similar messages [ 808.732436] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 16:58:12 (1785877092) [ 808.810283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 808.835745] LNet: 33336:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 808.853383] LNetError: 33336:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 808.879511] LNet: Removed LNI 192.168.201.5@tcp [ 808.913863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 808.948694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 808.965383] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 809.539907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp103 --gateway 192.168.201.[6-11/2]@tcp [ 809.554649] LNet: 33457:0:(router.c:729:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 809.560558] LNet: 33457:0:(router.c:729:lnet_add_route()) Skipped 4 previous similar messages [ 809.573039] LNetError: 33344:0:(lib-lnet.h:1274:lnet_set_route_aliveness()) route to tcp103 through 192.168.201.8@tcp has gone from up to down [ 809.584486] LNetError: 33344:0:(lib-lnet.h:1274:lnet_set_route_aliveness()) Skipped 4 previous similar messages [ 809.600748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route del --net tcp103 --gateway 192.168.201.[6-11/2]@tcp [ 817.334916] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 16:58:21 (1785877101) [ 817.439178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 817.469821] LNet: 34044:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 817.480319] LNetError: 34044:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 817.517265] LNet: Removed LNI 192.168.201.5@tcp [ 817.600824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 817.675666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking -1 [ 817.726958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 817.756995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 817.814934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 817.867588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 817.970235] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 818.113803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 0 [ 818.295611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 818.401418] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 818.506338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 818.643659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 818.683690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 818.776364] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 1 [ 818.912686] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 818.977215] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 819.114945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 819.259492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 819.315312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 819.387827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 2 [ 819.487724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 819.559680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 819.686469] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 819.825383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 819.868302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 819.977552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 820.126914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 820.166392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 820.248973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 820.333484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 820.376376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 820.434719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 4 [ 820.504569] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 820.546207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 820.641206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 828.202280] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 16:58:31 (1785877111) [ 830.135993] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_105 Need at least 2 remote nodes found oleg105-server [ 832.699768] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 16:58:36 (1785877116) [ 835.109775] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_106 Need at least 2 remote nodes found oleg105-server [ 837.379089] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 16:58:41 (1785877121) [ 837.661870] LNet: 35451:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 837.667995] LNet: 35451:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 11 previous similar messages [ 838.392719] Key type .llcrypt unregistered [ 838.394667] Key type ._llcrypt unregistered [ 848.276217] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 848.638557] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 848.652864] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 848.667361] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 851.497664] Key type ._llcrypt registered [ 851.500830] Key type .llcrypt registered [ 851.765693] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 851.814529] alg: No test for adler32 (adler32-zlib) [ 852.875040] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 852.968706] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 853.000793] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 853.901978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 853.924538] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 856.433711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 856.462225] LNet: Removed LNI 10.1.2.3@tcp [ 856.727757] LNet: 36177:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 856.744982] LNetError: 36177:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 856.785933] LNet: Removed LNI 192.168.201.5@tcp [ 858.069705] Key type .llcrypt unregistered [ 858.071395] Key type ._llcrypt unregistered [ 874.326233] Key type ._llcrypt registered [ 874.328772] Key type .llcrypt registered [ 875.932684] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 16:59:19 (1785877159) [ 876.850118] Key type .llcrypt unregistered [ 876.851698] Key type ._llcrypt unregistered [ 885.598082] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 885.877701] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 885.889700] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 885.899419] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 888.693564] Key type ._llcrypt registered [ 888.699065] Key type .llcrypt registered [ 889.067585] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 889.102894] alg: No test for adler32 (adler32-zlib) [ 890.200768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 890.297270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 890.336606] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 891.591607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 891.619687] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 894.831889] LNet: 37685:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 894.836170] LNetError: 37685:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 894.842268] LNet: Removed LNI 10.1.2.3@tcp [ 894.854391] LNet: Removed LNI 192.168.201.5@tcp [ 895.449501] Key type .llcrypt unregistered [ 895.453469] Key type ._llcrypt unregistered [ 910.350524] Key type ._llcrypt registered [ 910.354270] Key type .llcrypt registered [ 912.305847] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 16:59:55 (1785877195) [ 913.157260] Key type .llcrypt unregistered [ 913.158870] Key type ._llcrypt unregistered [ 922.611397] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 922.873574] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 922.893081] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 922.908239] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 925.574523] Key type ._llcrypt registered [ 925.583259] Key type .llcrypt registered [ 925.898142] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 925.945485] alg: No test for adler32 (adler32-zlib) [ 926.979214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 927.055251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 927.087239] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 929.722811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 929.768595] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 932.585490] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 932.606837] LNet: Removed LNI 10.1.2.3@tcp [ 932.623279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg:0 [ 932.662133] LNet: Removed LNI 10.1.2.31@tcp [ 932.677851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 932.713128] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 933.366280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 933.396978] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 934.536320] LNet: 39629:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 934.545404] LNetError: 39629:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg:0 not found [ 934.549653] LNet: Removed LNI 10.1.2.31@tcp [ 934.561564] LNet: Removed LNI 10.1.2.3@tcp [ 935.211484] Key type .llcrypt unregistered [ 935.221031] Key type ._llcrypt unregistered [ 955.872439] Key type ._llcrypt registered [ 955.873982] Key type .llcrypt registered [ 957.731756] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 17:00:40 (1785877240) [ 958.410030] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 958.448181] alg: No test for adler32 (adler32-zlib) [ 959.502094] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 959.547802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 959.576098] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 960.833430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 960.855789] LNet: 40587:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 960.865776] LNetError: 40587:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 960.881319] LNet: Removed LNI 192.168.201.5@tcp [ 960.903938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 960.982459] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --nid 192.168.201.5@tcp [ 961.023770] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 962.012440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --nid 192.168.201.5@tcp [ 962.037530] LNet: Removed LNI 192.168.201.5@tcp [ 962.151407] LNet: 40722:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 963.081568] Key type .llcrypt unregistered [ 963.083273] Key type ._llcrypt unregistered [ 979.038589] Key type ._llcrypt registered [ 979.042962] Key type .llcrypt registered [ 980.178935] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 17:01:04 (1785877264) [ 983.022568] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_111 Need at least 2 remote nodes found oleg105-server [ 984.961474] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 17:01:08 (1785877268) [ 985.609256] Key type .llcrypt unregistered [ 985.611212] Key type ._llcrypt unregistered [ 993.991723] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 994.225179] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 994.232474] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 994.237546] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 996.954357] Key type ._llcrypt registered [ 996.957721] Key type .llcrypt registered [ 997.144316] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 997.160433] alg: No test for adler32 (adler32-zlib) [ 998.204500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 998.291869] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 999.226294] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180] [ 1002.102378] LNet: 42583:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1002.121855] LNetError: 42583:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1002.135445] LNet: Removed LNI 192.168.201.5@tcp1 [ 1002.162722] LNet: Removed LNI 10.1.2.3@tcp2 [ 1003.385361] Key type .llcrypt unregistered [ 1003.388931] Key type ._llcrypt unregistered [ 1019.312521] Key type ._llcrypt registered [ 1019.314807] Key type .llcrypt registered [ 1020.555328] Lustre: DEBUG MARKER: == sanity-lnet test 150: Check import of ip2nets yaml sequence ========================================================== 17:01:44 (1785877304) [ 1021.175141] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1021.207223] alg: No test for adler32 (adler32-zlib) [ 1022.306862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1022.416815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1022.466369] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1024.023959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1024.055689] LNet: 43558:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1024.076558] LNetError: 43558:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1024.105699] LNet: Removed LNI 192.168.201.5@tcp [ 1024.151618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1024.273571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet.yaml [ 1024.314838] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1034.034105] Lustre: DEBUG MARKER: == sanity-lnet test 151: Check correct application of multiple ip2nets rules ========================================================== 17:01:57 (1785877317) [ 1034.213353] LNet: 44143:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1034.234523] LNetError: 44143:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1034.269234] LNet: Removed LNI 192.168.201.5@tcp [ 1035.211534] Key type .llcrypt unregistered [ 1035.214507] Key type ._llcrypt unregistered [ 1043.036277] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1043.213023] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1043.218116] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1043.225627] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1046.091601] Key type ._llcrypt registered [ 1046.095426] Key type .llcrypt registered [ 1046.407313] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1046.434197] alg: No test for adler32 (adler32-zlib) [ 1047.448218] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1047.511789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1047.532865] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1048.615323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1048.649511] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1051.860764] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1051.892467] LNet: 44898:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1051.908912] LNetError: 44898:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1051.931249] LNet: Removed LNI 10.1.2.3@tcp [ 1051.944823] LNet: Removed LNI 192.168.201.5@tcp [ 1051.973199] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1052.145483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 1052.192693] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1052.209912] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1052.968258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1053.008155] LNet: 44926:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1053.018113] LNetError: 44926:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1053.023560] LNetError: 44926:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1053.029880] LNet: Removed LNI 10.1.2.3@tcp [ 1053.061653] LNet: Removed LNI 192.168.201.5@tcp [ 1053.087462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1053.166331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 1053.201233] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1053.207626] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1053.912254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1053.965540] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 1055.869939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1055.917367] LNet: 45088:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1055.939440] LNetError: 45088:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1055.954960] LNetError: 45088:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1055.972092] LNet: Removed LNI 10.1.2.3@tcp [ 1055.976375] LNet: Removed LNI 192.168.201.5@tcp [ 1056.009942] LNet: Removed LNI 192.168.201.5@tcp2 [ 1056.049076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1056.170820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 1056.222402] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1056.238340] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1056.252126] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 1071.018900] Lustre: DEBUG MARKER: == sanity-lnet test 152: Check import of ip2nets with wildcard IP patterns ========================================================== 17:02:33 (1785877353) [ 1071.802973] LNet: 45684:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1071.843090] LNetError: 45684:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1071.867664] LNetError: 45684:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 1071.879722] LNet: Removed LNI 10.1.2.3@tcp [ 1071.880698] LNet: Removed LNI 192.168.201.5@tcp [ 1071.937370] LNet: Removed LNI 192.168.201.5@tcp2 [ 1072.801904] Key type .llcrypt unregistered [ 1072.803744] Key type ._llcrypt unregistered [ 1082.817720] Key type ._llcrypt registered [ 1082.820692] Key type .llcrypt registered [ 1082.971516] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1082.983365] alg: No test for adler32 (adler32-zlib) [ 1084.055557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1084.138093] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1084.216659] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1085.482831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1085.514520] LNet: 46087:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1085.527930] LNetError: 46087:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1085.554249] LNet: Removed LNI 192.168.201.5@tcp [ 1085.582928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1085.711943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1085.766079] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1086.688665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1086.724971] LNet: 46117:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1086.751375] LNetError: 46117:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1086.772794] LNet: Removed LNI 192.168.201.5@tcp [ 1086.813566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1086.959971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1087.000786] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1087.838579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1087.868800] LNet: 46147:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1087.884785] LNetError: 46147:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1087.944274] LNet: Removed LNI 192.168.201.5@tcp [ 1087.999161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1088.114753] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1088.187516] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1088.911367] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1089.131932] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1089.142243] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1089.179337] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1091.864662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1091.946574] LNet: 46226:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1091.955033] LNetError: 46226:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1091.973832] LNet: Removed LNI 192.168.201.5@tcp [ 1092.037436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1092.228229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1092.314239] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1101.727792] Lustre: DEBUG MARKER: == sanity-lnet test 153: Check import of ip2nets with IP address ranges ========================================================== 17:03:05 (1785877385) [ 1102.345864] LNet: 46826:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1102.364442] LNetError: 46826:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1102.394846] LNet: Removed LNI 192.168.201.5@tcp [ 1102.961221] Key type .llcrypt unregistered [ 1102.963721] Key type ._llcrypt unregistered [ 1111.837260] Key type ._llcrypt registered [ 1111.860114] Key type .llcrypt registered [ 1112.197494] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1112.240577] alg: No test for adler32 (adler32-zlib) [ 1113.240211] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1113.356956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1113.408339] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1114.204423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1114.220865] LNet: 47229:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1114.224793] LNetError: 47229:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1114.237287] LNet: Removed LNI 192.168.201.5@tcp [ 1114.252104] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1114.311730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-153.yaml [ 1114.336820] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1121.943671] Lustre: DEBUG MARKER: == sanity-lnet test 154: Check import of ip2nets with stepped IP address ranges ========================================================== 17:03:25 (1785877405) [ 1122.374644] LNet: 47822:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1122.380163] LNetError: 47822:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1122.392215] LNet: Removed LNI 192.168.201.5@tcp [ 1123.184464] Key type .llcrypt unregistered [ 1123.187124] Key type ._llcrypt unregistered [ 1132.764206] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1132.955271] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1132.971409] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1132.986929] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1135.740497] Key type ._llcrypt registered [ 1135.746981] Key type .llcrypt registered [ 1136.191408] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1136.223473] alg: No test for adler32 (adler32-zlib) [ 1137.333680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1137.452074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1137.491039] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1139.128889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1139.146577] LNet: 48271:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1139.158395] LNetError: 48271:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1139.177246] LNet: Removed LNI 192.168.201.5@tcp [ 1139.203514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1139.286488] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-154.yaml [ 1139.330249] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1147.160716] Lustre: DEBUG MARKER: == sanity-lnet test 155: Check ip2nets import failure with no matching rules ========================================================== 17:03:50 (1785877430) [ 1147.546955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1147.564899] LNet: 48870:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1147.570407] LNetError: 48870:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1147.583790] LNet: Removed LNI 192.168.201.5@tcp [ 1147.646815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1147.796497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-155.yaml [ 1155.960358] Lustre: DEBUG MARKER: == sanity-lnet test 156: Check import of ip2nets with comma-separated IP ranges ========================================================== 17:03:59 (1785877439) [ 1156.250221] LNet: 49454:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1156.863699] Key type .llcrypt unregistered [ 1156.867343] Key type ._llcrypt unregistered [ 1165.673502] Key type ._llcrypt registered [ 1165.679971] Key type .llcrypt registered [ 1166.158210] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1166.193676] alg: No test for adler32 (adler32-zlib) [ 1167.247210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1167.342552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1167.404092] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1168.819803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1168.856445] LNet: 49856:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1168.875058] LNetError: 49856:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1168.897681] LNet: Removed LNI 192.168.201.5@tcp [ 1168.933273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1169.027478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-156.yaml [ 1169.089850] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1178.619564] Lustre: DEBUG MARKER: == sanity-lnet test 157: Check import failure with malformed ip2nets YAML ========================================================== 17:04:21 (1785877461) [ 1179.175974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1179.217190] LNet: 50451:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1179.226645] LNetError: 50451:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1179.258798] LNet: Removed LNI 192.168.201.5@tcp [ 1179.317711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1179.491166] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-157.yaml [ 1189.242042] Lustre: DEBUG MARKER: == sanity-lnet test 158: Check import of ip2nets with single IPv6 address ========================================================== 17:04:32 (1785877472) [ 1191.832757] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_158 Interface has no IPv6 [ 1193.792022] Lustre: DEBUG MARKER: == sanity-lnet test 159: Check import of ip2nets with IPv6 /128 CIDR notation ========================================================== 17:04:37 (1785877477) [ 1195.021527] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_159 Interface has no IPv6 [ 1197.064683] Lustre: DEBUG MARKER: == sanity-lnet test 160: Check correct application of multiple IPv6 ip2nets rules ========================================================== 17:04:41 (1785877481) [ 1199.157257] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_160 Interface has no IPv6 [ 1201.089705] Lustre: DEBUG MARKER: == sanity-lnet test 161: Check import of ip2nets with various IPv6 prefix lengths ========================================================== 17:04:44 (1785877484) [ 1203.860373] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_161 Interface has no IPv6 [ 1206.206766] Lustre: DEBUG MARKER: == sanity-lnet test 162: Check ip2nets import failure with non-matching IPv6 ========================================================== 17:04:49 (1785877489) [ 1208.168874] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_162 Interface has no IPv6 [ 1211.162321] Lustre: DEBUG MARKER: == sanity-lnet test 170: Check CPTs and tunables parsing when importing ========================================================== 17:04:54 (1785877494) [ 1211.471484] LNet: 52497:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1212.830734] Key type .llcrypt unregistered [ 1212.834869] Key type ._llcrypt unregistered [ 1223.254529] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1223.539269] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1223.551499] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1223.563603] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1226.256521] Key type ._llcrypt registered [ 1226.262857] Key type .llcrypt registered [ 1226.579194] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1226.627772] alg: No test for adler32 (adler32-zlib) [ 1227.600932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1227.681511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1227.702542] LNet: Added LNI 192.168.201.5@tcp [8/512/0/180] [ 1227.710360] LNet: Added LNI 10.1.2.3@tcp [8/512/0/180] [ 1227.713614] LNet: 52831:0:(api-ni.c:2985:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1227.828212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1227.851181] LNet: 52850:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1227.856597] LNetError: 52850:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1227.861898] LNet: Removed LNI 10.1.2.3@tcp [ 1227.866858] LNetError: 52850:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1227.877964] LNet: Removed LNI 192.168.201.5@tcp [ 1227.916053] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1227.978732] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1228.003630] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1228.062147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1228.100337] LNet: 52874:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1228.117596] LNetError: 52874:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1228.140588] LNet: Removed LNI 192.168.201.5@tcp [ 1228.159954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1228.254070] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1228.294491] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1228.310178] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1228.315345] LNet: 52882:0:(api-ni.c:2985:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1228.486277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1228.530819] LNet: 52900:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1228.536208] LNetError: 52900:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1228.546153] LNet: Removed LNI 10.1.2.3@tcp [ 1228.548668] LNetError: 52900:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1228.570630] LNet: Removed LNI 192.168.201.5@tcp [ 1228.608127] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1228.696095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1228.738298] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1228.756713] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1228.762309] LNet: 52908:0:(api-ni.c:2985:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1229.053331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1229.097124] LNet: 52934:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1229.109083] LNetError: 52934:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1229.115951] LNet: Removed LNI 10.1.2.3@tcp [ 1229.123925] LNetError: 52934:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1229.146816] LNet: Removed LNI 192.168.201.5@tcp [ 1229.185849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1229.273082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1229.308616] LNet: Added LNI 192.168.201.5@tcp [8/256/0/182] [ 1229.314024] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 1229.440293] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1229.479892] LNet: 52962:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1229.490814] LNetError: 52962:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1229.502947] LNet: Removed LNI 10.1.2.3@tcp [ 1229.509387] LNetError: 52962:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1229.535062] LNet: Removed LNI 192.168.201.5@tcp [ 1229.562902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1229.657965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1229.684520] LNet: Added LNI 192.168.201.5@tcp [8/256/0/182] [ 1229.691507] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 1229.693441] LNet: 52970:0:(api-ni.c:2985:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1229.808929] LNet: Removed LNI 10.1.2.3@tcp [ 1229.830839] LNet: Removed LNI 192.168.201.5@tcp [ 1230.800680] Key type .llcrypt unregistered [ 1230.803884] Key type ._llcrypt unregistered [ 1248.230425] Key type ._llcrypt registered [ 1248.238078] Key type .llcrypt registered [ 1250.309246] Lustre: DEBUG MARKER: == sanity-lnet test 180: FastReg map error should not leak descriptors ========================================================== 17:05:33 (1785877533) [ 1252.693272] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_180 Need o2ib NETTYPE [ 1255.614742] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 17:05:38 (1785877538) [ 1256.325309] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1256.362228] alg: No test for adler32 (adler32-zlib) [ 1257.422330] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1257.520181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1257.566390] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1258.853727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --ip2net tcp 192.168.201.5 [ 1258.905411] LNet: 54241:0:(config.c:1738:lnet_parse_ip2nets()) Kernel parsing of ip2nets is deprecated. Consider converting to LNet YAML configuration. [ 1258.922191] LNet: Removed LNI 192.168.201.5@tcp [ 1266.922426] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 17:05:50 (1785877550) [ 1267.245944] LNet: 54812:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1267.258384] LNetError: 54812:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1267.286377] LNet: Removed LNI 192.168.201.5@tcp [ 1267.809535] Key type .llcrypt unregistered [ 1267.812388] Key type ._llcrypt unregistered [ 1276.276496] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1276.801952] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1278.674372] Key type ._llcrypt registered [ 1278.676816] Key type .llcrypt registered [ 1278.923834] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1278.944733] alg: No test for adler32 (adler32-zlib) [ 1280.392100] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1292.180364] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 17:06:14 (1785877574) [ 1292.422212] LNet: 55714:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1292.442631] LNetError: 55714:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1292.472233] LNet: Removed LNI 10.1.2.3@tcp [ 1293.846567] Key type .llcrypt unregistered [ 1293.849313] Key type ._llcrypt unregistered [ 1306.846106] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1307.236954] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1307.240808] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1307.255556] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1310.134731] Key type ._llcrypt registered [ 1310.137433] Key type .llcrypt registered [ 1310.395694] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1310.427380] alg: No test for adler32 (adler32-zlib) [ 1311.751642] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1321.561654] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 17:06:44 (1785877604) [ 1323.933885] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_202 kernel ip2nets deprecated in 2.17.53 [ 1327.074763] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 17:06:49 (1785877609) [ 1327.301166] LNet: 56911:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1327.320546] LNetError: 56911:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1327.353701] LNet: Removed LNI 10.1.2.3@tcp [ 1328.774296] Key type .llcrypt unregistered [ 1328.782681] Key type ._llcrypt unregistered [ 1339.642714] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1339.999417] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1340.004608] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1340.010360] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1342.983958] Key type ._llcrypt registered [ 1342.992181] Key type .llcrypt registered [ 1343.220126] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1343.243133] alg: No test for adler32 (adler32-zlib) [ 1344.237292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1344.370233] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1357.736486] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 17:07:20 (1785877640) [ 1358.449616] LNet: 58151:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1358.456980] LNetError: 58151:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1358.490914] LNet: Removed LNI 10.1.2.3@tcp [ 1359.481504] Key type .llcrypt unregistered [ 1359.483088] Key type ._llcrypt unregistered [ 1368.935475] Key type ._llcrypt registered [ 1368.941297] Key type .llcrypt registered [ 1369.371673] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1369.411517] alg: No test for adler32 (adler32-zlib) [ 1370.378472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1370.403116] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1380.565197] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1381.496318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_interrupt -r 1 [ 1381.544452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_interrupt -r 1 [ 1381.591881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1381.620272] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1381.663112] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1381.923863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1381.971822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1382.027455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1382.075191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1382.131781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1382.183475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1382.263189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1382.311271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1382.355651] LNet: 58594:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1382.391677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1382.468971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1382.552172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1382.625801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1382.688140] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1382.715017] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1382.774899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1382.837545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1383.566308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_dropped -r 1 [ 1383.617706] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_dropped -r 1 [ 1383.677390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1383.709424] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1384.029515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1384.073626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1384.114652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1384.156530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1384.203805] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1384.243777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1384.282083] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1384.317652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1384.345535] LNet: 58703:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1384.381354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1384.431775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1384.456565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1384.479530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1384.500720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1384.546867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1384.596425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1385.210391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_aborted -r 1 [ 1385.267519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_aborted -r 1 [ 1385.316242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1385.613652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1385.657326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1385.682326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1385.714127] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1385.744898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1385.785447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1385.813311] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1385.837956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1385.856686] LNet: 58812:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1385.882209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1385.912590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1385.943466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1385.978290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1386.018463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1386.067042] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1386.115363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1386.794635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_no_route -r 1 [ 1386.830514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_no_route -r 1 [ 1386.868931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1386.884481] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1386.892152] LNet: Skipped 1 previous similar message [ 1387.183742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1387.236856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1387.335883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1387.387894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1387.481592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1387.521431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1387.564486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1387.617648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1387.696277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1387.816600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1387.916694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1388.010406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1388.079285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1388.151709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1388.190682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1389.099720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_timeout -r 1 [ 1389.148099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_timeout -r 1 [ 1389.197064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1389.490721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1389.546243] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1389.599128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1389.645185] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1389.689503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1389.739082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1389.790616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1389.840280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1389.863968] LNet: 59031:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1389.868841] LNet: 59031:0:(api-ni.c:295:recovery_interval_set()) Skipped 1 previous similar message [ 1389.897443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1389.948947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1390.000368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1390.057128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1390.118976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1390.174266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1390.235933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1391.025837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_error -r 1 [ 1391.126402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_error -r 1 [ 1391.209401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1391.236851] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1391.244125] LNet: Skipped 1 previous similar message [ 1391.471644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1391.496995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1391.522674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1391.558657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1391.595946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1391.645195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1391.695676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1391.756521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1391.822040] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1391.892139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1392.005240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1392.103385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1392.184268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1392.227356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1392.301995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1392.839472] LNet: 59185:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1392.844711] LNetError: 59185:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1392.888060] LNet: Removed LNI 192.168.201.5@tcp [ 1393.734258] Key type .llcrypt unregistered [ 1393.738096] Key type ._llcrypt unregistered [ 1425.425498] Key type ._llcrypt registered [ 1425.437300] Key type .llcrypt registered [ 1428.075668] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 17:08:30 (1785877710) [ 1429.426450] Key type .llcrypt unregistered [ 1429.433063] Key type ._llcrypt unregistered [ 1440.371916] Key type ._llcrypt registered [ 1440.376288] Key type .llcrypt registered [ 1440.723244] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1440.785976] alg: No test for adler32 (adler32-zlib) [ 1441.912971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1441.950182] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1453.337189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1470.585374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1470.660393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1470.682593] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1472.481550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_interrupt -r 1 [ 1472.554170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_interrupt -r 1 [ 1472.628866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_interrupt -r 1 [ 1472.712238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_interrupt -r 1 [ 1472.809965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_interrupt -r 1 [ 1472.899258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_interrupt -r 1 [ 1472.972769] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_interrupt -r 1 [ 1473.057793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_interrupt -r 1 [ 1473.117106] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1473.148103] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1473.160413] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1473.585548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1473.637056] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1473.667974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1473.713881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1473.769502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1473.827464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1473.893964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1473.984684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1474.027641] LNet: 60663:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1474.069214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1474.166808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1474.209452] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1474.228711] LNet: Skipped 3 previous similar messages [ 1474.247783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1474.290384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1474.339506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1474.410219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1474.472494] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1475.172471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_dropped -r 1 [ 1475.215574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_dropped -r 1 [ 1475.249552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_dropped -r 1 [ 1475.291718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_dropped -r 1 [ 1475.348840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_dropped -r 1 [ 1475.390413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_dropped -r 1 [ 1475.425705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_dropped -r 1 [ 1475.463784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_dropped -r 1 [ 1475.503741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1475.523923] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1475.772477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1475.808420] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1475.840220] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1475.875452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1475.911435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1475.943827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1475.977473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1476.005885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1476.022240] LNet: 60784:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1476.036742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1476.068577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1476.095619] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1476.124804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1476.153111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1476.191661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1476.254814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1477.895061] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_aborted -r 1 [ 1478.016343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_aborted -r 1 [ 1478.135185] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_aborted -r 1 [ 1478.258628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_aborted -r 1 [ 1478.349223] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_aborted -r 1 [ 1478.441117] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_aborted -r 1 [ 1478.576736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_aborted -r 1 [ 1478.721623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_aborted -r 1 [ 1478.816308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1478.905661] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1478.915809] LNet: Skipped 2 previous similar messages [ 1479.322836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1479.392432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1479.445667] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1479.477674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1479.521545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1479.543934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1479.565637] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1479.594494] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1479.611505] LNet: 60905:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1479.628763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1479.655838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1479.686255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1479.725210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1479.759987] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1479.807961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1479.850437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1480.862271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_no_route -r 1 [ 1480.905698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_no_route -r 1 [ 1480.961519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_no_route -r 1 [ 1481.011027] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_no_route -r 1 [ 1481.054671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_no_route -r 1 [ 1481.086343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_no_route -r 1 [ 1481.113974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_no_route -r 1 [ 1481.145610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_no_route -r 1 [ 1481.173356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1481.420051] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1481.459073] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1481.526054] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1481.584602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1481.629698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1481.671244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1481.720669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1481.777756] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1481.813681] LNet: 61027:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1481.843535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1481.892753] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1481.945589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1481.992727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1482.058181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1482.107528] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1482.185386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1482.848972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_timeout -r 1 [ 1482.903633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_timeout -r 1 [ 1482.972166] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_timeout -r 1 [ 1483.057634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_timeout -r 1 [ 1483.152320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_timeout -r 1 [ 1483.184387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_timeout -r 1 [ 1483.210457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_timeout -r 1 [ 1483.235867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_timeout -r 1 [ 1483.271440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1483.286148] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1483.290178] LNet: Skipped 5 previous similar messages [ 1483.638715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1483.704592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1483.768868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1483.814855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1483.852640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1483.908744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1483.952383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1483.987148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1484.026964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1484.066588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1484.115945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1484.170695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1484.245936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1484.336985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1484.414551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1485.589573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_error -r 1 [ 1485.647788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_error -r 1 [ 1485.738541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_error -r 1 [ 1485.811952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_error -r 1 [ 1485.875521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_error -r 1 [ 1485.930933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_error -r 1 [ 1485.993873] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_error -r 1 [ 1486.103535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_error -r 1 [ 1486.195661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1486.576682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1486.615339] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1486.667694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1486.710755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1486.754889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1486.811113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1486.847975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1486.883434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1486.900625] LNet: 61269:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1486.904466] LNet: 61269:0:(api-ni.c:295:recovery_interval_set()) Skipped 1 previous similar message [ 1486.916723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1486.957463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1487.028264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1487.073913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1487.125201] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1487.187547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1487.254448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1495.355732] LNet: 61350:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1495.376443] LNetError: 61350:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1495.393563] LNet: Removed LNI 192.168.201.5@tcp [ 1496.423950] LNet: Removed LNI 192.168.201.5@tcp1 [ 1497.087744] Key type .llcrypt unregistered [ 1497.089898] Key type ._llcrypt unregistered [ 1523.670970] Key type ._llcrypt registered [ 1523.680566] Key type .llcrypt registered [ 1525.429649] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 17:10:08 (1785877808) [ 1526.792231] Key type .llcrypt unregistered [ 1526.793912] Key type ._llcrypt unregistered [ 1537.777381] Key type ._llcrypt registered [ 1537.788471] Key type .llcrypt registered [ 1538.450677] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1538.499504] alg: No test for adler32 (adler32-zlib) [ 1539.645599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1539.672956] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1555.098026] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1556.856461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_dropped -r 1 [ 1556.990649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_dropped -r 1 [ 1557.131040] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1557.179164] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1557.753901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1557.847196] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1557.930477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1558.011802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1558.127609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1558.229705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1558.292417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1558.331515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1558.356465] LNet: 62608:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1558.380711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1558.412185] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1558.457695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1558.520183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1558.590383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1558.616204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1558.640242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1559.029986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_error -r 1 [ 1559.049412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_error -r 1 [ 1559.068977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1559.079748] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1559.236129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1559.257091] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1559.277690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1559.300614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1559.322159] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1559.344450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1559.365476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1559.390231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1559.400519] LNet: 62717:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1559.413854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1559.431249] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1559.451627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1559.471531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1559.488849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1559.508870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1559.531136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1559.993194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_timeout -r 1 [ 1560.018425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_timeout -r 1 [ 1560.046352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1560.410378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1560.476719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1560.534090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1560.570584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1560.614670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1560.704885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1560.747358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1560.813281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1560.869503] LNet: 62826:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1560.905390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1560.962455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1561.011170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1561.067824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1561.145216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1561.205400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1561.286085] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1561.641645] LNet: 62871:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1561.653375] LNetError: 62871:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1561.673639] LNet: Removed LNI 192.168.201.5@tcp [ 1562.184396] Key type .llcrypt unregistered [ 1562.185837] Key type ._llcrypt unregistered [ 1587.322428] Key type ._llcrypt registered [ 1587.326666] Key type .llcrypt registered [ 1589.259041] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 17:11:12 (1785877872) [ 1590.391321] Key type .llcrypt unregistered [ 1590.392986] Key type ._llcrypt unregistered [ 1599.356411] Key type ._llcrypt registered [ 1599.358940] Key type .llcrypt registered [ 1599.512958] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1599.532466] alg: No test for adler32 (adler32-zlib) [ 1600.583584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1600.639707] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1611.496987] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1625.521936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1625.585485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1625.631919] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1627.859840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_dropped -r 1 [ 1627.933580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e remote_dropped -r 1 [ 1628.042556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_dropped -r 1 [ 1628.085846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e remote_dropped -r 1 [ 1628.149148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e remote_dropped -r 1 [ 1628.218957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e remote_dropped -r 1 [ 1628.286082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e remote_dropped -r 1 [ 1628.364490] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e remote_dropped -r 1 [ 1628.432656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1628.486402] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1628.491877] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.105@tcp [ 1628.846604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1628.915796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1628.984795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1629.047044] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1629.120479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1629.178707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1629.258425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1629.347797] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1629.387455] LNet: 64345:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1629.440201] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1629.497749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1629.539233] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1629.555271] LNet: Skipped 3 previous similar messages [ 1629.558847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1629.634373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1629.713444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1629.763105] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1629.876125] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1629.963604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1631.020917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_error -r 1 [ 1631.084043] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e remote_error -r 1 [ 1631.156537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_error -r 1 [ 1631.200673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e remote_error -r 1 [ 1631.280256] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e remote_error -r 1 [ 1631.347825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e remote_error -r 1 [ 1631.421858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e remote_error -r 1 [ 1631.492059] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e remote_error -r 1 [ 1631.542525] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1631.569591] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1631.905931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1631.979656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1632.051554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1632.108941] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1632.162908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1632.225140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1632.306247] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1632.377197] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1632.426619] LNet: 64468:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1632.459495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1632.529324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1632.602653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1632.668943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1632.745356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1632.844362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1632.938359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1632.996223] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1634.036350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_timeout -r 1 [ 1634.097188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e remote_timeout -r 1 [ 1634.161737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_timeout -r 1 [ 1634.245430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e remote_timeout -r 1 [ 1634.309863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e remote_timeout -r 1 [ 1634.384631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e remote_timeout -r 1 [ 1634.486471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e remote_timeout -r 1 [ 1634.622473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e remote_timeout -r 1 [ 1634.674354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.105@tcp [ 1634.725331] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1634.984614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1635.073666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1635.118367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1635.162890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1635.211790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1635.255948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1635.295867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1635.340431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1635.383508] LNet: 64592:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1635.413649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1635.473403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1635.536845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1635.572342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1635.613029] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1635.668625] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1635.722349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1635.755074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1643.153118] LNet: 64674:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1643.173735] LNetError: 64674:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1644.191622] LNet: Removed LNI 192.168.201.5@tcp [ 1644.193444] LNetError: 64674:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1644.217597] LNet: Removed LNI 192.168.201.5@tcp1 [ 1644.729759] Key type .llcrypt unregistered [ 1644.733857] Key type ._llcrypt unregistered [ 1667.085990] Key type ._llcrypt registered [ 1667.089647] Key type .llcrypt registered [ 1669.142148] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 17:12:31 (1785877951) [ 1671.009545] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_208 kernel ip2nets deprecated in 2.17.53 [ 1673.526463] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 17:12:36 (1785877956) [ 1674.982446] Key type .llcrypt unregistered [ 1674.985522] Key type ._llcrypt unregistered [ 1685.694253] Key type ._llcrypt registered [ 1685.699794] Key type .llcrypt registered [ 1686.274121] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1686.306249] alg: No test for adler32 (adler32-zlib) [ 1687.482257] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1687.543224] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1697.934148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1698.748188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e network_timeout -r 1 [ 1698.778961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e network_timeout -r 1 [ 1698.822271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1698.863763] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1699.160148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1699.203840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1699.232992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1699.273270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1699.311624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1699.355622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1699.398097] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1699.463961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1699.514969] LNet: 66224:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1699.546107] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1699.577336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1699.613676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1699.675514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1699.726729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1699.760951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1700.073983] LNet: 66267:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1700.083214] LNetError: 66267:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1700.110653] LNet: Removed LNI 192.168.201.5@tcp [ 1700.744421] Key type .llcrypt unregistered [ 1700.746538] Key type ._llcrypt unregistered [ 1729.235284] Key type ._llcrypt registered [ 1729.245390] Key type .llcrypt registered [ 1729.640050] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1729.691284] alg: No test for adler32 (adler32-zlib) [ 1730.894711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1730.941986] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1743.749316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1761.632298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1761.718581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1761.816597] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1765.196137] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e network_timeout -r 1 [ 1765.357671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e network_timeout -r 1 [ 1765.448993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e network_timeout -r 1 [ 1765.585883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e network_timeout -r 1 [ 1765.685614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e network_timeout -r 1 [ 1765.806750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e network_timeout -r 1 [ 1765.924861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e network_timeout -r 1 [ 1766.023854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e network_timeout -r 1 [ 1766.198065] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 1766.270319] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1766.819575] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1766.839784] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.105@tcp [ 1766.970574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1767.065724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1767.165601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1767.273354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1767.406371] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1767.557751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1767.619174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1767.704602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1767.737759] LNet: 67152:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1767.773524] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1767.844956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1767.887954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1767.904745] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1767.930815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1767.968220] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1768.029519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1774.212829] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1774.216318] LNet: Skipped 1 previous similar message [ 1774.525171] LNet: 67230:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1774.532274] LNetError: 67230:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1774.536906] LNet: Removed LNI 192.168.201.5@tcp [ 1774.558031] LNet: Removed LNI 192.168.201.5@tcp1 [ 1775.023764] Key type .llcrypt unregistered [ 1775.028026] Key type ._llcrypt unregistered [ 1800.867214] Key type ._llcrypt registered [ 1800.870161] Key type .llcrypt registered [ 1802.775262] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 17:14:45 (1785878085) [ 1803.291820] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1803.319273] alg: No test for adler32 (adler32-zlib) [ 1804.342996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1804.455440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1804.489044] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1805.582893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1805.615776] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1806.887638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 1806.996099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 1807.056321] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 1807.117465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 1807.176430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 1807.226792] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1807.265668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.5@tcp [ 1807.299803] LNetError: 68332:0:(api-ni.c:4472:lnet_ni_set_healthv()) manually adding local NI 192.168.201.5@tcp to recovery [ 1807.455218] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1807.455639] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1809.503427] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1809.914273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1813.599420] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1814.843832] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1814.896101] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1814.971046] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1815.007445] LNet: 68439:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1815.020752] LNetError: 68439:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1816.031345] LNet: Removed LNI 192.168.201.5@tcp [ 1816.034938] LNetError: 68439:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1816.070729] LNet: Removed LNI 192.168.201.5@tcp1 [ 1816.122725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1816.230621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1816.266363] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1817.218620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1817.234759] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1817.862262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 1817.886693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1817.923726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 1817.950479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 1817.975761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 1818.002562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 1818.025212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1818.048530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.5@tcp [ 1818.065266] LNetError: 68712:0:(api-ni.c:4472:lnet_ni_set_healthv()) manually adding local NI 192.168.201.5@tcp to recovery [ 1818.212841] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1820.568766] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1824.352038] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1824.356639] LNet: Skipped 1 previous similar message [ 1828.874720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1828.940035] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1828.998753] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1838.783518] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 17:15:21 (1785878121) [ 1838.903390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1838.942355] LNet: 69417:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1838.945389] LNetError: 69417:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1838.964641] LNet: Removed LNI 192.168.201.5@tcp [ 1839.016491] LNet: Removed LNI 192.168.201.5@tcp1 [ 1839.071176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1839.209272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1839.250406] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1840.360412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1840.385420] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1841.427320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 1841.531166] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 1841.578620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 1841.615503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 1841.657909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1841.718733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1841.740833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.5@tcp --health 0 [ 1841.822908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1842.207241] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.5@tcp [ 1842.209248] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1842.211632] LNet: Skipped 1 previous similar message [ 1857.656986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1857.911784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1857.973639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1858.026473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.5@tcp --health 500 [ 1858.189233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1865.149661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1865.265565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1865.319513] LNet: 69916:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1865.339309] LNetError: 69916:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1865.356189] LNetError: 69916:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1865.371391] LNet: Removed LNI 192.168.201.5@tcp [ 1865.380848] LNet: Removed LNI 192.168.201.5@tcp1 [ 1865.421248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1865.527691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1865.598245] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1866.669285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1866.694332] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 1868.082379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 1868.150314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1868.227338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 1868.274907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 1868.336638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 1868.386923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1868.462807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1868.542564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.5@tcp --health 0 [ 1869.607144] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 1869.623539] LNet: Skipped 3 previous similar messages [ 1870.515961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1880.571070] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1880.625109] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1880.709725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1890.646698] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 17:16:13 (1785878173) [ 1891.392130] LNet: 70954:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1891.417378] LNetError: 70954:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1891.430346] LNetError: 70954:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1891.452262] LNet: Removed LNI 192.168.201.5@tcp [ 1891.486828] LNet: Removed LNI 192.168.201.5@tcp1 [ 1892.455156] Key type .llcrypt unregistered [ 1892.461950] Key type ._llcrypt unregistered [ 1900.589441] Key type ._llcrypt registered [ 1900.590738] Key type .llcrypt registered [ 1900.889102] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1900.924542] alg: No test for adler32 (adler32-zlib) [ 1901.966751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1902.012685] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1912.526865] Lustre: DEBUG MARKER: Initial discovery [ 1912.586523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.105@tcp [ 1915.509623] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag [ 1916.067743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 -e local_error [ 1916.149917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.105@tcp [ 1916.192116] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1916.227037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1916.318134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d 192.168.201.105@tcp -r 1 -m GET -l 3 [ 1916.389038] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 1916.401803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.201.105@tcp -r 1 -m GET -e local_error [ 1916.462848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.201.105@tcp -r 1 -m PUT -l 6 [ 1918.798773] Lustre: DEBUG MARKER: Force oleg105-client.virtnet to discover 192.168.201.105@tcp (in background) [ 1918.884143] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.105@tcp [ 1921.140634] Lustre: DEBUG MARKER: Force oleg105-server to discover 192.168.201.5@tcp [ 1921.951597] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1924.367728] Lustre: DEBUG MARKER: Wait for 71712 [ 1925.344220] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 1926.559269] Lustre: DEBUG MARKER: Finished wait on 71712 [ 1926.601486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay del -a [ 1926.634804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1926.892046] LNet: 72174:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1926.896827] LNetError: 72174:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1926.914935] LNet: Removed LNI 192.168.201.5@tcp [ 1927.517517] Key type .llcrypt unregistered [ 1927.521055] Key type ._llcrypt unregistered [ 1948.454981] Key type ._llcrypt registered [ 1948.456579] Key type .llcrypt registered [ 1950.032915] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 17:17:13 (1785878233) [ 1950.839484] Key type .llcrypt unregistered [ 1950.843662] Key type ._llcrypt unregistered [ 1962.619130] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1963.064479] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1963.088342] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1963.106057] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1966.427832] Key type ._llcrypt registered [ 1966.430407] Key type .llcrypt registered [ 1966.669392] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1966.698877] alg: No test for adler32 (adler32-zlib) [ 1967.693349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1967.756869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1967.786349] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1969.087583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1969.201618] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1980.818450] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 17:17:43 (1785878263) [ 1981.001240] LNet: 74251:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1981.008744] LNetError: 74251:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1981.011141] LNet: Removed LNI 10.1.2.3@tcp [ 1981.051050] LNet: Removed LNI 192.168.201.5@tcp [ 1982.058606] Key type .llcrypt unregistered [ 1982.060556] Key type ._llcrypt unregistered [ 1995.028625] Key type ._llcrypt registered [ 1995.030079] Key type .llcrypt registered [ 1995.400929] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1995.439342] alg: No test for adler32 (adler32-zlib) [ 1996.685246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1996.831795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1996.896492] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 1998.457996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1998.486788] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2000.293035] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 10.1.2.3@tcp 192.168.201.5@tcp [ 2001.443750] LNetError: Unexpected error -22 connecting to 192.168.201.5@tcp at host 192.168.201.5:988 [ 2009.662321] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 17:18:13 (1785878293) [ 2009.988823] LNet: 75395:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2010.000156] LNetError: 75395:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2010.012601] LNet: Removed LNI 10.1.2.3@tcp [ 2010.030063] LNet: Removed LNI 192.168.201.5@tcp [ 2010.570397] Key type .llcrypt unregistered [ 2010.576197] Key type ._llcrypt unregistered [ 2020.123376] Key type ._llcrypt registered [ 2020.128678] Key type .llcrypt registered [ 2020.996780] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2021.070477] alg: No test for adler32 (adler32-zlib) [ 2022.345491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2022.478940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2022.557566] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 2023.821736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2023.871910] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2025.440435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 192.168.201.5@tcp1 --nid 192.168.201.5@tcp2 [ 2033.386752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp1 192.168.201.5@tcp2 [ 2033.442074] LNetError: 76236:0:(lib-move.c:2101:lnet_handle_find_routed_path()) Unable to send message from 192.168.201.5@tcp1 to 192.168.201.5@tcp2 - Route table may be misconfigured [ 2033.502680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp2 192.168.201.5@tcp1 [ 2046.059190] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 17:18:49 (1785878329) [ 2046.631151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2046.666662] LNet: 76811:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2046.687990] LNetError: 76811:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2046.703679] LNet: Removed LNI 192.168.201.5@tcp1 [ 2047.721150] LNet: Removed LNI 192.168.201.5@tcp2 [ 2047.811252] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2047.920331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2047.957869] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2049.138409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2049.169688] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 2050.162377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2050.246604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.5@tcp [ 2050.301491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp1 192.168.201.5@tcp1 [ 2050.374885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.201.5@tcp -d 192.168.201.5@tcp -e network_timeout [ 2050.424590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -e network_timeout [ 2050.501571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -e network_timeout [ 2050.591353] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -e network_timeout [ 2050.652649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2050.707042] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 2050.725768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 2050.868462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 2050.977777] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 2051.017798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 2059.924165] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 17:19:02 (1785878342) [ 2059.979609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2060.002149] LNet: 77665:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2060.008209] LNetError: 77665:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2060.012817] LNetError: 77665:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2060.020187] LNet: Removed LNI 192.168.201.5@tcp [ 2061.040206] LNet: Removed LNI 192.168.201.5@tcp1 [ 2061.093466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2061.243651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 0@lo [ 2061.683788] LNet: 77714:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2062.306585] Key type .llcrypt unregistered [ 2062.307847] Key type ._llcrypt unregistered [ 2077.774379] Key type ._llcrypt registered [ 2077.780688] Key type .llcrypt registered [ 2079.277962] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 17:19:22 (1785878362) [ 2080.090175] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2080.118161] alg: No test for adler32 (adler32-zlib) [ 2081.151541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2082.682992] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces [ 2084.527612] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 17:19:28 (1785878368) [ 2084.943438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2084.964269] LNet: 78813:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2085.000453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2085.074338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2085.099733] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2085.691380] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2085.704824] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 2086.538261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2086.615878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp1 [ 2086.688906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp1 [ 2096.114493] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 17:19:39 (1785878379) [ 2098.622656] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_220 Need at least 2 remote nodes found oleg105-server [ 2101.838403] Lustre: DEBUG MARKER: == sanity-lnet test 221: Add routes w/DD disabled - check aliveness ========================================================== 17:19:44 (1785878384) [ 2104.604692] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_221 Need at least 2 remote nodes found oleg105-server [ 2107.200473] Lustre: DEBUG MARKER: == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 17:19:50 (1785878390) [ 2110.304232] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_222 Need at least 2 remote nodes found oleg105-server [ 2112.852550] Lustre: DEBUG MARKER: == sanity-lnet test 223: Check avoid_asym_router_failure=1 w/DD disabled ========================================================== 17:19:56 (1785878396) [ 2115.542000] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_223 Need at least 2 remote nodes found oleg105-server [ 2117.770953] Lustre: DEBUG MARKER: == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 17:20:01 (1785878401) [ 2120.307517] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_224 Need at least 2 remote nodes found oleg105-server [ 2122.930308] Lustre: DEBUG MARKER: == sanity-lnet test 225: Check avoid_asym_router_failure=0 w/DD disabled ========================================================== 17:20:06 (1785878406) [ 2124.863348] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_225 Need at least 2 remote nodes found oleg105-server [ 2126.742833] Lustre: DEBUG MARKER: == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 17:20:10 (1785878410) [ 2129.004906] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_226 Need at least 3 remote nodes found oleg105-server [ 2130.954476] Lustre: DEBUG MARKER: == sanity-lnet test 227: Check router peer health w/DD disabled ========================================================== 17:20:14 (1785878414) [ 2133.144255] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_227 Need at least 3 remote nodes found oleg105-server [ 2135.897691] Lustre: DEBUG MARKER: == sanity-lnet test 228: Routes should stay up when health is decremented ========================================================== 17:20:18 (1785878418) [ 2138.288536] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_228 Need at least 3 remote nodes found oleg105-server [ 2141.686710] Lustre: DEBUG MARKER: == sanity-lnet test 230: Test setting conns-per-peer ===== 17:20:24 (1785878424) [ 2141.931969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2142.000536] LNet: 82692:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2142.012383] LNetError: 82692:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2142.021778] LNet: Removed LNI 192.168.201.5@tcp [ 2143.078292] LNet: Removed LNI 192.168.201.5@tcp1 [ 2143.208389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2143.379079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2143.464651] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2144.871706] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 4 [ 2145.057328] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2145.518064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2145.625279] LNet: 82840:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2145.629984] LNetError: 82840:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2145.634239] LNetError: 82840:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2145.659931] LNet: Removed LNI 192.168.201.5@tcp [ 2145.695391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2145.779801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 5 [ 2145.808276] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2147.121882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2147.497448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2147.560041] LNetError: 82986:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2148.583797] LNet: Removed LNI 192.168.201.5@tcp [ 2148.685826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2148.914095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2149.010880] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2150.487075] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 6 [ 2150.704590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2151.048821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2151.082673] LNet: 83133:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2151.089701] LNet: 83133:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 2151.098826] LNetError: 83133:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2151.111958] LNet: Removed LNI 192.168.201.5@tcp [ 2151.134437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2151.181741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 7 [ 2151.242698] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2152.107120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2152.268603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2153.321689] LNet: Removed LNI 192.168.201.5@tcp [ 2153.357163] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2153.446150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2153.487740] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2154.474396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 8 [ 2154.559285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2154.847164] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2155.954848] LNet: Removed LNI 192.168.201.5@tcp [ 2156.014732] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2156.130214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 9 [ 2156.176147] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2157.998576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2158.392398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2158.412139] LNet: 83571:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2158.420097] LNet: 83571:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 2158.425973] LNetError: 83571:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2158.430697] LNetError: 83571:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 2158.442781] LNet: Removed LNI 192.168.201.5@tcp [ 2158.467684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2158.532959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2158.556681] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2159.367498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 10 [ 2159.418772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2159.550423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2160.614741] LNet: Removed LNI 192.168.201.5@tcp [ 2160.651310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2160.736844] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 11 [ 2160.751312] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2161.236216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2162.708803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2163.820290] LNet: Removed LNI 192.168.201.5@tcp [ 2163.883026] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2164.068620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2164.147593] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2165.636608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 12 [ 2165.694755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2165.867649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2166.951206] LNet: Removed LNI 192.168.201.5@tcp [ 2166.987778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2167.099833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 13 [ 2167.155394] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2168.511690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2168.907164] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2168.936133] LNet: 84166:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2168.945996] LNet: 84166:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 2168.961062] LNetError: 84166:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2168.967795] LNetError: 84166:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 2170.025373] LNet: Removed LNI 192.168.201.5@tcp [ 2170.055220] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2170.144434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2170.183846] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2171.728189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 14 [ 2171.901048] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2173.535338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2174.635868] LNet: Removed LNI 192.168.201.5@tcp [ 2174.673689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2174.784236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 15 [ 2174.846834] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2176.125504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2176.354588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2177.396782] LNet: Removed LNI 192.168.201.5@tcp [ 2177.444680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2177.538962] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2177.574297] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2178.256740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 128 [ 2178.315342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2178.343523] LNet: Removed LNI 192.168.201.5@tcp [ 2178.360378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2178.409376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2178.438483] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2179.293305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer -1 [ 2179.362689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2179.394754] LNet: Removed LNI 192.168.201.5@tcp [ 2179.409759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2179.451884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 128 [ 2179.511179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2179.535441] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2179.573603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 1000000000000000000000000 [ 2179.644803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2179.679339] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2179.739590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer foo [ 2179.801936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2179.831938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2179.865206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 8 [ 2179.882590] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2180.842621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2180.901846] LNet: Removed LNI 192.168.201.5@tcp [ 2180.950407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2181.058387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 2191.486598] Lustre: DEBUG MARKER: == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 17:21:14 (1785878474) [ 2191.859793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2191.909046] LNet: 85489:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2191.916654] LNet: 85489:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 8 previous similar messages [ 2191.963506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2192.076547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 [ 2192.126376] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/180] [ 2194.324724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2194.357650] LNetError: 85628:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2194.366932] LNetError: 85628:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 2194.403910] LNet: Removed LNI 192.168.201.5@tcp231 [ 2194.454480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2194.655440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 2194.702025] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/180] [ 2194.710776] LNet: 85636:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2195.486581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 2195.552106] LNet: Removed LNI 192.168.201.5@tcp231 [ 2195.588067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=-1 [ 2195.656330] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/180] [ 2197.557824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2197.600900] LNet: Removed LNI 192.168.201.5@tcp231 [ 2197.627896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2197.723510] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 2197.759895] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/0] [ 2197.768512] LNet: 85788:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2198.184286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 2198.209211] LNet: Removed LNI 192.168.201.5@tcp231 [ 2198.228908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=0 [ 2198.272481] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/0] [ 2199.858535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2199.927735] LNet: Removed LNI 192.168.201.5@tcp231 [ 2199.982485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2200.145190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 2200.203863] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/60] [ 2200.215404] LNet: 85940:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2201.433378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 2201.495485] LNet: Removed LNI 192.168.201.5@tcp231 [ 2201.508477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=60 [ 2201.589855] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/60] [ 2203.145048] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2203.216187] LNet: Removed LNI 192.168.201.5@tcp231 [ 2203.255314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2203.368292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 2203.395658] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/180] [ 2203.404268] LNet: 86091:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2204.319678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 2204.386850] LNet: Removed LNI 192.168.201.5@tcp231 [ 2204.410994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=180 [ 2204.475972] LNet: Added LNI 192.168.201.5@tcp231 [8/256/0/180] [ 2218.458516] Lustre: DEBUG MARKER: == sanity-lnet test 232: Test setting ToS value ========== 17:21:40 (1785878500) [ 2218.681063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2218.758209] LNet: Removed LNI 192.168.201.5@tcp231 [ 2218.787170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2218.905185] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 [ 2218.951424] LNet: Added LNI 192.168.201.5@tcp232 [8/256/0/180] [ 2220.419228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 2220.502843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 2220.535582] LNet: Removed LNI 192.168.201.5@tcp232 [ 2220.574270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos -1 [ 2220.613046] LNet: Added LNI 192.168.201.5@tcp232 [8/256/0/180] [ 2222.358323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 2222.471229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 2222.552038] LNet: Removed LNI 192.168.201.5@tcp232 [ 2222.593737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 104 [ 2222.704434] LNet: Added LNI 192.168.201.5@tcp232 [8/256/0/180] [ 2225.126483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 2225.210944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 2225.296151] LNet: Removed LNI 192.168.201.5@tcp232 [ 2225.356345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 106 [ 2225.471683] LNet: Added LNI 192.168.201.5@tcp232 [8/256/0/180] [ 2226.861642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 2226.926706] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 2226.972228] LNet: Removed LNI 192.168.201.5@tcp232 [ 2226.990712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 300 [ 2236.873578] Lustre: DEBUG MARKER: == sanity-lnet test 233: Check for successful resends ==== 17:21:59 (1785878519) [ 2237.715472] LNet: 87932:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2237.729165] LNet: 87932:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 2239.303484] Key type .llcrypt unregistered [ 2239.311395] Key type ._llcrypt unregistered [ 2251.478199] Key type ._llcrypt registered [ 2251.479876] Key type .llcrypt registered [ 2251.980475] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2252.039753] alg: No test for adler32 (adler32-zlib) [ 2253.379128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2253.448917] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2264.088896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 2281.150687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2281.267164] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2281.331741] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 2285.821329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_interrupt -r 2 [ 2285.879657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_interrupt -r 2 [ 2285.959867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_interrupt -r 2 [ 2286.020442] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_interrupt -r 2 [ 2286.059759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_interrupt -r 2 [ 2286.111834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_interrupt -r 2 [ 2286.200970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_interrupt -r 2 [ 2286.274294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_interrupt -r 2 [ 2286.431326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2286.507863] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 2286.550204] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 2287.109580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2287.160656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2287.234919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2287.302791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2287.376175] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2287.433749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2287.521503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2287.601074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2287.652579] LNet: 88601:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2287.694176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2287.755822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2287.811659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2287.878789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2287.949228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2288.016921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2288.755296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_dropped -r 2 [ 2288.809982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_dropped -r 2 [ 2288.883811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_dropped -r 2 [ 2288.945200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_dropped -r 2 [ 2289.018478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_dropped -r 2 [ 2289.081193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_dropped -r 2 [ 2289.119653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_dropped -r 2 [ 2289.189905] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_dropped -r 2 [ 2289.225134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2289.260956] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 2289.678570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2289.732265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2289.790482] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2289.853305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2289.909105] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2289.957468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2290.019506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2290.069390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2290.149034] LNet: 88722:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2290.206746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2290.269679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2290.374752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2290.453408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2290.550583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2290.660962] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2291.988620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_aborted -r 2 [ 2292.055600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_aborted -r 2 [ 2292.104579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_aborted -r 2 [ 2292.149067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_aborted -r 2 [ 2292.216381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_aborted -r 2 [ 2292.293079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_aborted -r 2 [ 2292.351192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_aborted -r 2 [ 2292.414544] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_aborted -r 2 [ 2292.452852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2292.485826] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 2292.825847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2292.879271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2292.929156] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2292.992242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2293.040349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2293.085763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2293.150319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2293.198983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2293.235462] LNet: 88842:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2293.267276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2293.308705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2293.345656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2293.378506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2293.438251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2293.516450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2294.485516] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_no_route -r 2 [ 2294.576633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_no_route -r 2 [ 2294.670899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_no_route -r 2 [ 2294.759601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_no_route -r 2 [ 2294.863651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_no_route -r 2 [ 2294.989638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_no_route -r 2 [ 2295.098421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_no_route -r 2 [ 2295.190129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_no_route -r 2 [ 2295.305501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2295.345656] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 2295.824877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2295.867113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2295.918647] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2295.965782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2296.015815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2296.063777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2296.147477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2296.236557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2296.263496] LNet: 88963:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2296.303787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2296.370209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2296.424380] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2296.479683] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2296.534885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2296.583967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2297.623579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e local_timeout -r 2 [ 2297.695186] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e local_timeout -r 2 [ 2297.769958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e local_timeout -r 2 [ 2297.854322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e local_timeout -r 2 [ 2297.957606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e local_timeout -r 2 [ 2298.047587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e local_timeout -r 2 [ 2298.169111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e local_timeout -r 2 [ 2298.256566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e local_timeout -r 2 [ 2298.394306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2298.899390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2298.962443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2299.031577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2299.080379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2299.144724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2299.234382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2299.348237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2299.401638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2299.492431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2299.573554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2299.654831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2299.719277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2299.793231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2299.851639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2300.690063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp -m GET -e remote_dropped -r 2 [ 2300.790031] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.5@tcp -d 192.168.201.105@tcp1 -m GET -e remote_dropped -r 2 [ 2300.881962] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp -m GET -e remote_dropped -r 2 [ 2300.951538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp -d 192.168.201.5@tcp1 -m GET -e remote_dropped -r 2 [ 2301.039686] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp -m GET -e remote_dropped -r 2 [ 2301.144853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.105@tcp1 -m GET -e remote_dropped -r 2 [ 2301.232823] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp -m GET -e remote_dropped -r 2 [ 2301.320893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.5@tcp1 -d 192.168.201.5@tcp1 -m GET -e remote_dropped -r 2 [ 2301.418941] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.5@tcp 192.168.201.105@tcp [ 2301.460710] LNet: There was an unexpected network error while writing to 192.168.201.105: rc = -22 [ 2301.471065] LNet: Skipped 1 previous similar message [ 2301.483763] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.105@tcp [ 2301.911566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2302.000316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2302.070322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2302.124527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2302.237436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2302.321316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2302.403764] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2302.517451] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2302.552342] LNet: 89203:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2302.563435] LNet: 89203:0:(api-ni.c:295:recovery_interval_set()) Skipped 1 previous similar message [ 2302.599070] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2302.654458] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2302.713303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2302.775427] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2302.825521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2302.901439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2310.612462] LNet: 89282:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2310.620629] LNetError: 89282:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2310.634486] LNet: Removed LNI 192.168.201.5@tcp [ 2310.643817] LNet: Removed LNI 192.168.201.5@tcp1 [ 2311.275659] Key type .llcrypt unregistered [ 2311.279785] Key type ._llcrypt unregistered [ 2334.280827] Key type ._llcrypt registered [ 2334.282665] Key type .llcrypt registered [ 2335.832316] Lustre: DEBUG MARKER: == sanity-lnet test 234: Verify NI status changes with manual health value transitions ========================================================== 17:23:39 (1785878619) [ 2336.362182] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2336.391172] alg: No test for adler32 (adler32-zlib) [ 2337.360943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2337.419152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2337.445927] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2338.377609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 0 [ 2338.410675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.5@tcp [ 2338.427624] LNetError: 90264:0:(api-ni.c:4472:lnet_ni_set_healthv()) manually adding local NI 192.168.201.5@tcp to recovery [ 2338.570913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --nid 192.168.201.5@tcp [ 2338.767761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2346.921591] Lustre: DEBUG MARKER: == sanity-lnet test 235: Verify NI status DOWN when health reaches 0 after failures ========================================================== 17:23:50 (1785878630) [ 2347.204842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2347.238883] LNet: 90853:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2347.244838] LNetError: 90853:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2347.287508] LNet: Removed LNI 192.168.201.5@tcp [ 2347.306711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2347.380217] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2347.398038] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2347.978573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2347.993394] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 2348.879684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 100 --nid 192.168.201.5@tcp [ 2348.899342] LNetError: 91120:0:(api-ni.c:4472:lnet_ni_set_healthv()) manually adding local NI 192.168.201.5@tcp to recovery [ 2349.407381] LNet: 1 local NIs in recovery (showing 1): 192.168.201.5@tcp [ 2349.408674] LNet: There was an unexpected network error while writing to 192.168.201.5: rc = -22 [ 2350.228251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 900 --nid 192.168.201.5@tcp [ 2361.383416] Lustre: DEBUG MARKER: == sanity-lnet test 236: Local NI state propagates to routers ========================================================== 17:24:05 (1785878645) [ 2364.581627] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_236 Need at least 2 remote nodes found oleg105-server [ 2366.992976] Lustre: DEBUG MARKER: == sanity-lnet test 237: Check MR forwarding feature ===== 17:24:10 (1785878650) [ 2369.704432] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_237 Need at least 2 remote nodes found oleg105-server [ 2372.031328] Lustre: DEBUG MARKER: == sanity-lnet test 238: Check MR forwarding feature skipped for non-MR source ========================================================== 17:24:15 (1785878655) [ 2374.702661] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_238 Need at least 2 remote nodes found oleg105-server [ 2377.362406] Lustre: DEBUG MARKER: == sanity-lnet test 241: Check setting LND timeout value via lnetctl updates tunables ========================================================== 17:24:20 (1785878660) [ 2377.990610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2378.037279] LNet: 92751:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2378.049298] LNetError: 92751:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2378.066124] LNet: Removed LNI 192.168.201.5@tcp [ 2378.088235] LNet: Removed LNI 192.168.201.5@tcp1 [ 2378.143665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2378.229445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2378.255357] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2378.996742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set -h [ 2379.072745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 51 [ 2379.126189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 2379.204664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 0 [ 2379.244225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 2379.285650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 50 [ 2388.908661] Lustre: DEBUG MARKER: == sanity-lnet test 245: Check read-only params do not appear in export --backup ========================================================== 17:24:31 (1785878671) [ 2389.081141] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2389.131433] LNet: 93475:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2389.142427] LNetError: 93475:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2389.153469] LNetError: 93475:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2389.173158] LNet: Removed LNI 192.168.201.5@tcp [ 2389.263195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2389.374773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2389.432505] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2397.909054] Lustre: DEBUG MARKER: == sanity-lnet test 250: test that linux routes are added ========================================================== 17:24:41 (1785878681) [ 2398.308121] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2398.345471] LNet: 94182:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2398.353454] LNetError: 94182:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2398.367258] LNet: Removed LNI 192.168.201.5@tcp [ 2398.401443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2398.499447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2398.549586] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2407.527487] Lustre: DEBUG MARKER: == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 17:24:51 (1785878691) [ 2409.505755] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_251 Need kfi NETTYPE [ 2412.023337] Lustre: DEBUG MARKER: == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 17:24:55 (1785878695) [ 2412.659491] LNet: 95219:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2412.665893] LNetError: 95219:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2412.699710] LNet: Removed LNI 192.168.201.5@tcp [ 2413.533586] Key type .llcrypt unregistered [ 2413.535119] Key type ._llcrypt unregistered [ 2422.889398] Key type ._llcrypt registered [ 2422.892991] Key type .llcrypt registered [ 2423.184943] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2423.206908] alg: No test for adler32 (adler32-zlib) [ 2424.143716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2424.171205] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2434.447393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 2441.546816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --timeout 15 192.168.201.105@tcp [ 2441.996467] LNet: 95646:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2442.008593] LNetError: 95646:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2442.045678] LNet: Removed LNI 192.168.201.5@tcp [ 2442.667490] Key type .llcrypt unregistered [ 2442.670808] Key type ._llcrypt unregistered [ 2457.650712] Key type ._llcrypt registered [ 2457.652756] Key type .llcrypt registered [ 2459.492957] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253 [ 2461.853781] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254 [ 2463.574471] Lustre: DEBUG MARKER: == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 17:25:47 (1785878747) [ 2465.611901] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_255 kernel ip2nets deprecated in 2.17.53 [ 2467.754212] Lustre: DEBUG MARKER: == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 17:25:51 (1785878751) [ 2469.811945] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_256 Need at least 2 remote nodes found oleg105-server [ 2471.934133] Lustre: DEBUG MARKER: == sanity-lnet test 257: Test graceful router shutdown/startup ========================================================== 17:25:55 (1785878755) [ 2474.022483] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_257 Need at least 3 remote nodes found oleg105-server [ 2476.035079] Lustre: DEBUG MARKER: == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 17:25:59 (1785878759) [ 2478.102281] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2478.349513] alg: No test for adler32 (adler32-zlib) [ 2479.610708] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 2479.986893] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2481.751166] Key type lgssc registered [ 2483.222484] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2505.217779] Key type lgssc unregistered [ 2505.558180] LNet: 98825:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2505.569984] LNetError: 98825:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2505.601719] LNet: Removed LNI 192.168.201.5@tcp [ 2506.441188] Key type .llcrypt unregistered [ 2506.443185] Key type ._llcrypt unregistered [ 2525.693765] Key type ._llcrypt registered [ 2525.696288] Key type .llcrypt registered [ 2526.809106] Lustre: DEBUG MARKER: == sanity-lnet test 265: Import of invalid CPT should fail ========================================================== 17:26:50 (1785878810) [ 2527.903320] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2527.916867] alg: No test for adler32 (adler32-zlib) [ 2528.850684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2528.922328] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2528.993414] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2529.046496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2529.106883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2538.588563] Lustre: DEBUG MARKER: == sanity-lnet test 266: Validate CPT parsing in network sequence ========================================================== 17:27:01 (1785878821) [ 2538.767864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2538.831213] LNet: 100236:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2538.872409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2538.961409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2538.980337] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2540.634257] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2540.688385] LNet: 100386:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2540.702311] LNetError: 100386:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2540.723379] LNet: Removed LNI 192.168.201.5@tcp [ 2540.762108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2540.847453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 2540.878195] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2541.883310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2541.941747] LNet: 100415:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2541.963989] LNetError: 100415:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2541.990375] LNet: Removed LNI 192.168.201.5@tcp [ 2542.054354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2542.248193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 2542.334702] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2551.085891] Lustre: DEBUG MARKER: == sanity-lnet test 270: Test NID metadata =============== 17:27:14 (1785878834) [ 2551.476831] LNet: 101006:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2551.487898] LNetError: 101006:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2551.504902] LNet: Removed LNI 192.168.201.5@tcp [ 2552.427600] Key type .llcrypt unregistered [ 2552.431432] Key type ._llcrypt unregistered [ 2561.038236] Key type ._llcrypt registered [ 2561.045925] Key type .llcrypt registered [ 2561.299396] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2561.348165] alg: No test for adler32 (adler32-zlib) [ 2562.379272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2562.545496] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2562.553043] LNet: *** cfs_fail_loc=e003, val=100*** [ 2563.837870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2563.867345] LNet: *** cfs_fail_loc=e003, val=100*** [ 2563.871593] LNet: Skipped 2 previous similar messages [ 2563.901468] Ping reply received with mapping: [ 2563.905859] found metadata reply NI[192.168.201.5@tcp] [ 2563.910331] 05 c9 a8 c0 00 00 02 00 5a 5a 5a 5a 5a 5a 5a 5a [ 2563.916070] 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a [ 2563.921732] 5a 5a 5a 5a 5a 5a 5a 5a [ 2564.250097] LNet: 101409:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2564.260309] LNetError: 101409:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2564.285515] LNet: Removed LNI 192.168.201.5@tcp [ 2565.171464] Key type .llcrypt unregistered [ 2565.173235] Key type ._llcrypt unregistered [ 2581.286380] Key type ._llcrypt registered [ 2581.289025] Key type .llcrypt registered [ 2583.008845] Lustre: DEBUG MARKER: == sanity-lnet test 280: Don't panic when request_module fails ========================================================== 17:27:46 (1785878866) [ 2584.244747] Key type .llcrypt unregistered [ 2584.248176] Key type ._llcrypt unregistered [ 2593.049320] Key type ._llcrypt registered [ 2593.056807] Key type .llcrypt registered [ 2593.567126] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2593.589168] alg: No test for adler32 (adler32-zlib) [ 2594.766180] LNetError: 102469:0:(api-ni.c:2849:lnet_load_lnd()) Can't load LND tcp, module ksocklnd, rc=256 [ 2595.524583] Key type .llcrypt unregistered [ 2595.530864] Key type ._llcrypt unregistered [ 2602.732260] Key type ._llcrypt registered [ 2602.733958] Key type .llcrypt registered [ 2604.420085] Lustre: DEBUG MARKER: == sanity-lnet test 290: Check that lnet acceptor is using explicit address binding ========================================================== 17:28:07 (1785878887) [ 2605.665236] Key type .llcrypt unregistered [ 2605.674692] Key type ._llcrypt unregistered [ 2615.498994] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2615.512909] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2618.271116] Key type ._llcrypt registered [ 2618.273403] Key type .llcrypt registered [ 2618.591903] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2618.636693] alg: No test for adler32 (adler32-zlib) [ 2619.751779] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2619.852442] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2631.290540] Lustre: DEBUG MARKER: == sanity-lnet test 291: load lnet using custom accept port for bulk tcp connections ========================================================== 17:28:34 (1785878914) [ 2631.600859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2631.625765] LNet: 104260:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2631.630793] LNetError: 104260:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2631.659581] LNet: Removed LNI 10.1.2.3@tcp [ 2631.692252] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2633.056562] Key type .llcrypt unregistered [ 2633.062227] Key type ._llcrypt unregistered [ 2641.696625] Key type ._llcrypt registered [ 2641.700919] Key type .llcrypt registered [ 2641.955494] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2641.990758] alg: No test for adler32 (adler32-zlib) [ 2643.016079] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2643.168547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2644.610447] LNet: 104562:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2644.626917] LNetError: 104562:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2644.655662] LNet: Removed LNI 192.168.201.5@tcp [ 2646.193468] Key type .llcrypt unregistered [ 2646.197137] Key type ._llcrypt unregistered [ 2663.970125] Key type ._llcrypt registered [ 2663.974757] Key type .llcrypt registered [ 2665.911644] Lustre: DEBUG MARKER: == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 17:29:09 (1785878949) [ 2667.266748] Key type .llcrypt unregistered [ 2667.268237] Key type ._llcrypt unregistered [ 2677.835337] Key type ._llcrypt registered [ 2677.838609] Key type .llcrypt registered [ 2678.128337] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2678.149503] alg: No test for adler32 (adler32-zlib) [ 2689.576167] Lustre: DEBUG MARKER: == sanity-lnet test 301: Check for dynamic adds of same/wrong interface (memory leak) ========================================================== 17:29:31 (1785878971) [ 2689.717478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2689.816023] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2689.991761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2690.095948] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2691.638806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2691.734673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if ens2 [ 2691.780887] LNet: Removed LNI 192.168.201.5@tcp [ 2692.429700] LNet: 106323:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2693.072622] Key type .llcrypt unregistered [ 2693.077807] Key type ._llcrypt unregistered [ 2710.503627] Key type ._llcrypt registered [ 2710.507605] Key type .llcrypt registered [ 2712.574669] Lustre: DEBUG MARKER: == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 17:29:55 (1785878995) [ 2713.670902] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2713.706869] alg: No test for adler32 (adler32-zlib) [ 2714.759684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2714.846146] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2714.878143] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2715.891812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2715.961929] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug peer --nid 192.168.201.5@tcp [ 2715.996660] LNet: Conn 0 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 2716.005606] LNet: Conn 0 rx [scheduled, ready, state]: 0, 0, 1 [ 2716.010684] LNet: Conn 0 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2715, 0, 2765 [ 2716.025417] LNet: Conn 0 sched [nconns, cpt]: 2, 0 [ 2716.026846] LNet: Conn 1 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 2716.036303] LNet: Conn 1 rx [scheduled, ready, state]: 0, 0, 1 [ 2716.045842] LNet: Conn 1 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2715, 0, 2765 [ 2716.056977] LNet: Conn 1 sched [nconns, cpt]: 2, 0 [ 2726.456214] Lustre: DEBUG MARKER: == sanity-lnet test 303: Check peer NI health after link down ========================================================== 17:30:10 (1785879010) [ 2727.561235] LNet: 107867:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2727.577175] LNetError: 107867:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2727.596601] LNet: Removed LNI 192.168.201.5@tcp [ 2728.408672] Key type .llcrypt unregistered [ 2728.410367] Key type ._llcrypt unregistered [ 2736.094216] Key type ._llcrypt registered [ 2736.098513] Key type .llcrypt registered [ 2736.373444] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2736.400306] alg: No test for adler32 (adler32-zlib) [ 2737.351700] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2737.385371] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2749.847196] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 2763.551795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2763.646720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2763.716899] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 2765.494723] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 2765.850477] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2765.870954] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2765.884362] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 2768.491434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp99 --if test1pg [ 2768.536811] LNet: Added LNI 10.1.2.3@tcp99 [8/256/0/180] [ 2781.005592] Lustre: DEBUG MARKER: == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 17:31:05 (1785879065) [ 2781.533759] LNet: 109382:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2781.544276] LNetError: 109382:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2781.555670] LNet: Removed LNI 192.168.201.5@tcp [ 2782.623840] LNet: Removed LNI 192.168.201.5@tcp1 [ 2782.625508] LNetError: 109382:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2782.637305] LNetError: 109382:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2782.672261] LNet: Removed LNI 10.1.2.3@tcp99 [ 2783.434609] Key type .llcrypt unregistered [ 2783.438426] Key type ._llcrypt unregistered [ 2792.267662] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2792.284395] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2795.303421] Key type ._llcrypt registered [ 2795.306957] Key type .llcrypt registered [ 2795.578791] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2795.603992] alg: No test for adler32 (adler32-zlib) [ 2796.600988] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2796.691710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2796.737042] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2797.549612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2797.589494] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2800.098521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.1.2.3@tcp --lock_prim [ 2800.122863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2800.145987] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 10.1.2.3@tcp --state 1048577 [ 2800.155462] LNetError: 110132:0:(peer.c:2362:lnet_del_peer_ni()) peer 10.1.2.3@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 2800.172181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2800.196287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2800.681229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2800.768194] LNet: 110152:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2800.778570] LNetError: 110152:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2800.785557] LNet: Removed LNI 10.1.2.3@tcp [ 2801.829274] LNet: Removed LNI 192.168.201.5@tcp [ 2802.773483] Key type .llcrypt unregistered [ 2802.778573] Key type ._llcrypt unregistered [ 2823.251326] Key type ._llcrypt registered [ 2823.255021] Key type .llcrypt registered [ 2825.342526] Lustre: DEBUG MARKER: == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 17:31:48 (1785879108) [ 2826.184373] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2826.224130] alg: No test for adler32 (adler32-zlib) [ 2827.321452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2827.433123] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2827.481804] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2828.818616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 2829.002329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping oleg105-client@tcp [ 2837.905103] Lustre: DEBUG MARKER: == sanity-lnet test 310: Set timeout and verify ========== 17:32:01 (1785879121) [ 2838.051705] LNet: 111683:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2838.067941] LNetError: 111683:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2838.097926] LNet: Removed LNI 192.168.201.5@tcp [ 2839.116352] Key type .llcrypt unregistered [ 2839.118371] Key type ._llcrypt unregistered [ 2849.996221] Key type ._llcrypt registered [ 2849.999194] Key type .llcrypt registered [ 2850.256118] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2850.269425] alg: No test for adler32 (adler32-zlib) [ 2851.142810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure --all [ 2851.176085] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2851.246741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 2861.754759] Lustre: DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 17:32:25 (1785879145) [ 2863.776939] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type [ 2867.150710] Lustre: DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 17:32:29 (1785879149) [ 2869.543773] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type [ 2871.522827] Lustre: DEBUG MARKER: == sanity-lnet test 350: Check import --exec of ping/discover ========================================================== 17:32:35 (1785879155) [ 2871.762735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2871.810598] LNet: 113115:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2871.815262] LNetError: 113115:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2871.828047] LNet: Removed LNI 192.168.201.5@tcp [ 2871.841601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2871.919429] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2871.956655] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2872.695444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 2872.723170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 2881.131088] Lustre: DEBUG MARKER: == sanity-lnet test 400: Check for udsp add/delete net rule without net num ========================================================== 17:32:44 (1785879164) [ 2881.662860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2881.729339] LNet: 113826:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2881.745055] LNetError: 113826:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2882.795473] LNet: Removed LNI 192.168.201.5@tcp [ 2882.861309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2882.973938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --priority 0 [ 2883.040598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2883.538606] LNet: 113866:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2884.099373] Key type .llcrypt unregistered [ 2884.107291] Key type ._llcrypt unregistered [ 2901.376891] Key type ._llcrypt registered [ 2901.384897] Key type .llcrypt registered [ 2903.192710] Lustre: DEBUG MARKER: == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 17:33:06 (1785879186) [ 2904.191799] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2904.210631] alg: No test for adler32 (adler32-zlib) [ 2905.298635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2905.431357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2905.507519] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2907.082391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2907.197576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2918.112179] Lustre: DEBUG MARKER: == sanity-lnet test 402: Destination net rule should not panic ========================================================== 17:33:21 (1785879201) [ 2918.273735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2918.317337] LNet: 115383:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2918.339915] LNetError: 115383:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2918.400549] LNet: Removed LNI 192.168.201.5@tcp [ 2918.478858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2918.625295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst kfi --priority 0 [ 2918.671223] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 402@kfi [ 2926.824750] Lustre: DEBUG MARKER: == sanity-lnet test 403: Add and delete udsp net and nid rules ========================================================== 17:33:30 (1785879210) [ 2927.022183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2927.076181] LNet: 115963:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2927.123544] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2927.287509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2927.847793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2927.901289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2928.012465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src 1.1.1.1@tcp --prio 1 [ 2928.751492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2928.800826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2928.924921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2928.975427] LNet: 116001:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2928.999890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2929.111925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2929.729395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2929.771746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2929.815357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst 1.1.1.1@tcp --prio 1 [ 2930.305430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2930.394512] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2943.232936] Lustre: DEBUG MARKER: == sanity-lnet test 404: Check udsp src/dst net rule ===== 17:33:45 (1785879225) [ 2943.390310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2943.437929] LNet: 116595:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2943.498320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2943.697816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2943.785510] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2945.846355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2945.900424] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2948.022368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2948.313878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2948.869729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2948.928871] LNet: 116882:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2948.947530] LNetError: 116882:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2948.963735] LNet: Removed LNI 192.168.201.5@tcp [ 2949.003936] LNet: Removed LNI 192.168.201.5@tcp2 [ 2949.142465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2949.413121] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2949.429486] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2949.443982] LNet: 116888:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2950.437766] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2950.772064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2950.845919] LNetError: 116925:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2950.849422] LNetError: 116925:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2950.869868] LNet: Removed LNI 192.168.201.5@tcp [ 2950.929450] LNet: Removed LNI 192.168.201.5@tcp2 [ 2950.993984] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2951.053290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2951.077120] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2952.156445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2952.202958] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2953.588775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2953.663699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 2953.803224] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2954.199422] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2954.232260] LNetError: 117210:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2954.239119] LNetError: 117210:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2954.262900] LNet: Removed LNI 192.168.201.5@tcp [ 2955.323956] LNet: Removed LNI 192.168.201.5@tcp2 [ 2955.403870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2955.533438] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2955.548894] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2955.558801] LNet: 117216:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2956.346857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2956.629231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2956.653764] LNet: Removed LNI 192.168.201.5@tcp [ 2957.677884] LNet: Removed LNI 192.168.201.5@tcp2 [ 2957.776685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2957.916856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2957.974278] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2959.150097] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2959.179977] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2960.196053] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2960.238718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2960.499324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2960.515560] LNet: 117542:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2960.519207] LNet: 117542:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 2960.525670] LNetError: 117542:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2960.533273] LNetError: 117542:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 2960.539331] LNet: Removed LNI 192.168.201.5@tcp [ 2960.559573] LNet: Removed LNI 192.168.201.5@tcp2 [ 2960.597189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2960.702542] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2960.719080] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2960.725720] LNet: 117548:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2961.362895] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2961.646619] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2962.719278] LNet: Removed LNI 192.168.201.5@tcp [ 2962.764291] LNet: Removed LNI 192.168.201.5@tcp2 [ 2962.799919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2962.957214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2963.008723] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2964.091668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2964.112529] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2965.391245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2965.476558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 2965.539172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2965.756839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2966.815504] LNet: Removed LNI 192.168.201.5@tcp [ 2967.847888] LNet: Removed LNI 192.168.201.5@tcp2 [ 2967.882160] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2968.008090] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2968.012090] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2968.023683] LNet: 117876:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2968.619458] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2968.797459] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2968.815896] LNetError: 117913:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2968.821989] LNetError: 117913:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 2968.827840] LNet: Removed LNI 192.168.201.5@tcp [ 2969.893807] LNet: Removed LNI 192.168.201.5@tcp2 [ 2969.927702] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2970.043698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2970.097347] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2971.101408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2971.116307] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2972.373674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2972.421071] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2972.500118] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2973.002801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2974.047159] LNet: Removed LNI 192.168.201.5@tcp [ 2974.056759] LNet: Removed LNI 192.168.201.5@tcp2 [ 2974.142591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2974.344734] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2974.356737] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2974.367433] LNet: 118216:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2975.339132] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2975.984036] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2977.057626] LNet: Removed LNI 192.168.201.5@tcp [ 2977.080794] LNet: Removed LNI 192.168.201.5@tcp2 [ 2977.150064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2977.246981] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2977.297028] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2978.558281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2978.599479] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2979.823544] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2979.895515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 2979.952537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 2980.041554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2980.700275] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2980.729798] LNet: 118571:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2980.735598] LNet: 118571:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 2981.791348] LNet: Removed LNI 192.168.201.5@tcp [ 2982.890498] LNet: Removed LNI 192.168.201.5@tcp2 [ 2982.950352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2983.050990] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2983.064205] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 2983.074706] LNet: 118577:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2984.012972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 2993.386722] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_410 skipping ALWAYS excluded test 410 [ 2995.557390] Lustre: DEBUG MARKER: == sanity-lnet test 450: Check import of multiple interfaces ========================================================== 17:34:39 (1785879279) [ 2995.975120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2996.001541] LNetError: 119336:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2996.014886] LNetError: 119336:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 7 previous similar messages [ 2996.032141] LNet: Removed LNI 192.168.201.5@tcp [ 2996.053056] LNet: Removed LNI 192.168.201.5@tcp2 [ 2996.095197] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2996.545846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2996.581279] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2997.214927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2997.235990] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2998.421561] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2998.460611] LNet: Removed LNI 10.1.2.3@tcp [ 2998.472690] LNet: Removed LNI 192.168.201.5@tcp [ 2998.520153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2998.631602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-450.yaml [ 2998.682775] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 2998.691769] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2999.278847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del /tmp/sanity-lnet-450.yaml [ 2999.318439] LNet: Removed LNI 192.168.201.5@tcp [ 2999.332630] LNet: Removed LNI 10.1.2.3@tcp [ 3006.694568] Lustre: DEBUG MARKER: == sanity-lnet test 475: per-operation latency stats populate and reset ========================================================== 17:34:50 (1785879290) [ 3006.924128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 3007.027958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 3007.128940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 3007.146165] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 3007.654530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 3007.692387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 3007.732104] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 3007.774158] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 3007.810749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 3007.839856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 3007.864555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 3007.887234] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 3007.912307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 3007.937267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 3007.961936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.5@tcp [ 3008.070216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 3016.583482] Lustre: DEBUG MARKER: == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 17:35:00 (1785879300) [ 3017.063216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 3017.112873] LNet: 120949:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3017.117693] LNet: 120949:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 3018.156026] LNet: Removed LNI 192.168.201.5@tcp [ 3018.209979] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 3018.328405] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 3018.589514] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 3018.594896] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 3018.601770] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 3020.950649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 3020.995870] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 3022.114556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 3022.143394] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 3026.201167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 3026.396314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d *@tcp -r 1 -l 1 -m PUT [ 3026.573595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 192.168.201.5@tcp -m PUT -e local_timeout -r 1 [ 3026.777881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 10.1.2.3@tcp -m PUT -e local_timeout -r 1 [ 3026.979485] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 3026.991182] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 3027.661938] LNetError: 121013:0:(lib-move.c:4801:lnet_parse()) 192.168.201.5@tcp, src 192.168.201.5@tcp: Dropping PUT (error -108 looking up sender) [ 3028.710892] LNetError: 121476:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 3028.721146] LNetError: 121476:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 4 previous similar messages [ 3028.735484] LNet: Removed LNI 10.1.2.3@tcp [ 3029.798950] LNet: Removed LNI 192.168.201.5@tcp [ 3030.447489] Key type .llcrypt unregistered [ 3030.450767] Key type ._llcrypt unregistered [ 3048.138362] Key type ._llcrypt registered [ 3048.144814] Key type .llcrypt registered [ 3049.882328] Lustre: DEBUG MARKER: == sanity-lnet test 501: Verify lnetctl peer set --health (SR) ========================================================== 17:35:33 (1785879333) [ 3050.329228] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3050.345942] alg: No test for adler32 (adler32-zlib) [ 3051.241439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 3051.631962] LNet: 122328:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3052.133962] Key type .llcrypt unregistered [ 3052.143333] Key type ._llcrypt unregistered [ 3061.597678] Key type ._llcrypt registered [ 3061.599729] Key type .llcrypt registered [ 3062.085376] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 3062.123241] alg: No test for adler32 (adler32-zlib) [ 3063.262821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 3063.368465] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 3073.223574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 3073.469026] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 3074.009514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.105@tcp [ 3074.192339] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 3074.220820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 3074.242883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 3074.276933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 3074.314209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 3074.364829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 3074.416932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 3074.465182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 3074.479421] LNet: 122782:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 3074.513855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 3074.559610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 3074.602749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 3074.650780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 3074.655766] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.105@tcp [ 3074.684982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 3074.737381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 3075.372745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 3075.594697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 3075.627953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 3075.655242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 3075.691994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 3075.726987] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 3075.770966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 3075.805169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 3075.846107] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 3075.879787] LNet: 122900:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 3075.926386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 3075.991144] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 3076.051100] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 3076.156114] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 3076.256977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 3076.336773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 3077.296210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.105@tcp [ 3077.667161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 3077.689642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 3077.712642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 3077.762713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 3077.793455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 3077.837587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 3077.895818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 3077.949685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 3077.984638] LNet: 123017:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 3078.025808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 3078.098371] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 3078.150319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 3078.200659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 3078.253551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 3078.298098] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 3079.276231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 3079.563947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 3079.616733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 3079.661401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 3079.704586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 3079.750138] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 3079.824645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 3079.874587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 3079.940956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 3079.998660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 3080.053422] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 3080.124842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 3080.177489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 3080.253256] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 3080.319259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 3080.691829] LNet: 123184:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3080.697122] LNetError: 123184:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3081.795302] LNet: Removed LNI 192.168.201.5@tcp [ 3082.533359] Key type .llcrypt unregistered [ 3082.542169] Key type ._llcrypt unregistered [ 3106.699329] Key type ._llcrypt registered [ 3106.701376] Key type .llcrypt registered [ 3108.232160] Lustre: DEBUG MARKER: == sanity-lnet test 502: Verify lnetctl peer set --health (MR) ========================================================== 17:36:31 (1785879391) [ 3108.582677] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3108.592510] alg: No test for adler32 (adler32-zlib) [ 3109.620271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 3110.053306] LNet: 124073:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3110.786416] Key type .llcrypt unregistered [ 3110.788295] Key type ._llcrypt unregistered [ 3120.043610] Key type ._llcrypt registered [ 3120.048984] Key type .llcrypt registered [ 3120.428908] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 3120.485039] alg: No test for adler32 (adler32-zlib) [ 3121.511575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 3121.554749] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 3131.009718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.105@tcp [ 3142.809657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 3142.870993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 3142.901035] LNet: Added LNI 192.168.201.5@tcp1 [8/256/0/180] [ 3143.622417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 3144.253961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.105@tcp [ 3144.640607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 3144.701564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 3144.766526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 3144.808564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 3144.889602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 3144.945298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 3144.988810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 3145.052982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 3145.090683] LNet: 124730:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 3145.120204] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.105@tcp [ 3145.131313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 3145.166579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 3145.188976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 3145.211558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 3145.261909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 3145.312957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 3145.892105] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 3146.066388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 3146.087851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 3146.107940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 3146.130833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 3146.155884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 3146.183794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 3146.211206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 3146.240699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 3146.252459] LNet: 124847:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 3146.265581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 3146.285621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 3146.310835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 3146.331491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 3146.355626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 3146.377369] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 3146.950652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.105@tcp [ 3147.160898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 3147.182868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 3147.204962] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 3147.243139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 3147.291890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 3147.342288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 3147.378675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 3147.443574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 3147.482571] LNet: 124964:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 3147.520898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 3147.602686] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 3147.677464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 3147.761337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 3147.819909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 3147.888750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 3148.869622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 3149.195324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 3149.241427] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 3149.283761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 3149.333627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 3149.383571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 3149.444533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 3149.485959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 3149.529309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 3149.559978] LNet: 125081:0:(api-ni.c:295:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 3149.585619] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 3149.649471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 3149.706834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 3149.761398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 3149.807415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 3149.848798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 3156.323731] LNet: 125166:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3156.338856] LNetError: 125166:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3157.407268] LNet: Removed LNI 192.168.201.5@tcp [ 3157.410509] LNetError: 125166:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3157.430452] LNet: Removed LNI 192.168.201.5@tcp1 [ 3158.125251] Key type .llcrypt unregistered [ 3158.128935] Key type ._llcrypt unregistered [ 3182.305817] Key type ._llcrypt registered [ 3182.309424] Key type .llcrypt registered [ 3184.560624] Lustre: DEBUG MARKER: == sanity-lnet test 525: keep reachable peer on local net delete ========================================================== 17:37:47 (1785879467) [ 3185.456659] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3185.465573] alg: No test for adler32 (adler32-zlib) [ 3186.409555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 3186.465407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 3186.500151] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 3187.371225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 3187.395085] LNet: Added LNI 192.168.201.5@tcp2 [8/256/0/180] [ 3188.833627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.5@tcp [ 3189.054157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp [ 3190.177092] LNet: Removed LNI 192.168.201.5@tcp [ 3201.406765] Lustre: DEBUG MARKER: == sanity-lnet test 550: DD-off: keep route up when gw reachable via another local NI ========================================================== 17:38:04 (1785879484) [ 3203.599581] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_550 Need at least 2 local interfaces for multi-rail [ 3206.363112] Lustre: DEBUG MARKER: == sanity-lnet test 600: lnet_legacy2yaml: basic ip2nets conversion ========================================================== 17:38:09 (1785879489) [ 3215.748655] Lustre: DEBUG MARKER: == sanity-lnet test 601: lnet_legacy2yaml: basic routes conversion ========================================================== 17:38:19 (1785879499) [ 3227.339538] Lustre: DEBUG MARKER: == sanity-lnet test 602: lnet_legacy2yaml: routes with priority ========================================================== 17:38:29 (1785879509) [ 3239.422933] Lustre: DEBUG MARKER: == sanity-lnet test 603: lnet_legacy2yaml: default hop flag ========================================================== 17:38:42 (1785879522) [ 3248.787783] Lustre: DEBUG MARKER: == sanity-lnet test 604: lnet_legacy2yaml: default hop with explicit value ========================================================== 17:38:52 (1785879532) [ 3259.836298] Lustre: DEBUG MARKER: == sanity-lnet test 605: lnet_legacy2yaml: quote leading star in IP pattern ========================================================== 17:39:03 (1785879543) [ 3271.273523] Lustre: DEBUG MARKER: == sanity-lnet test 606: lnet_legacy2yaml: file inputs === 17:39:13 (1785879553) [ 3284.288527] Lustre: DEBUG MARKER: == sanity-lnet test 607: lnet_legacy2yaml: options line with combined ip2nets and routes ========================================================== 17:39:26 (1785879566) [ 3297.126349] Lustre: DEBUG MARKER: == sanity-lnet test 608: lnet_legacy2yaml: error on invalid rule (missing IP pattern) ========================================================== 17:39:39 (1785879579) [ 3306.938280] Lustre: DEBUG MARKER: == sanity-lnet test 609: lnet_legacy2yaml: error on no input ========================================================== 17:39:50 (1785879590) [ 3316.172944] Lustre: DEBUG MARKER: == sanity-lnet test 610: lnet_legacy2yaml: multiple files and inline, check order ========================================================== 17:39:59 (1785879599) [ 3324.658140] Lustre: DEBUG MARKER: == sanity-lnet test 611: lnet_legacy2yaml: invalid hopcount ========================================================== 17:40:08 (1785879608) [ 3339.079701] Lustre: DEBUG MARKER: == sanity-lnet test 612: lnet_legacy2yaml: malformed IP pattern ========================================================== 17:40:20 (1785879620) [ 3350.733429] Lustre: DEBUG MARKER: == sanity-lnet test 613: lnet_legacy2yaml: multiple IP patterns in single rule ========================================================== 17:40:33 (1785879633) [ 3361.426502] Lustre: DEBUG MARKER: == sanity-lnet test 614: lnet_legacy2yaml: interfaces enumeration with multiple IP ranges ========================================================== 17:40:44 (1785879644) [ 3372.292678] Lustre: DEBUG MARKER: == sanity-lnet test 615: lnet_legacy2yaml: modprobe configuration file input ========================================================== 17:40:55 (1785879655) [ 3381.237578] Lustre: DEBUG MARKER: == sanity-lnet test 616: lnet_legacy2yaml: combining modprobe file with explicit ip2nets file ========================================================== 17:41:05 (1785879665) [ 3391.560585] Lustre: DEBUG MARKER: == sanity-lnet test 617: lnet_legacy2yaml: round-trip tunables from LNet to modprobe to YAML ========================================================== 17:41:15 (1785879675) [ 3392.022524] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 3392.058761] LNet: 136871:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3392.071027] LNetError: 136871:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3392.092887] LNet: Removed LNI 192.168.201.5@tcp2 [ 3392.123456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 3392.308759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 3392.354996] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 3393.721242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 3393.742843] LNet: 137013:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3393.747810] LNetError: 137013:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3393.763345] LNet: Removed LNI 192.168.201.5@tcp [ 3393.778795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 3393.848624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 3393.875947] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 3401.726308] Lustre: DEBUG MARKER: == sanity-lnet test 618: lnet_legacy2yaml: alternative route format ========================================================== 17:41:25 (1785879685) [ 3410.348522] Lustre: DEBUG MARKER: == sanity-lnet test 619: lnet_legacy2yaml: multiple alternative format routes from file ========================================================== 17:41:34 (1785879694) [ 3420.989640] Lustre: DEBUG MARKER: == sanity-lnet test 620: lnet_legacy2yaml: mixed legacy and alternative route formats ========================================================== 17:41:44 (1785879704) [ 3433.647445] Lustre: DEBUG MARKER: == sanity-lnet test 621: lnet_legacy2yaml: alternative format with invalid hopcount ========================================================== 17:41:56 (1785879716) [ 3442.878901] Lustre: DEBUG MARKER: == sanity-lnet test 622: lnet_legacy2yaml: alternative format with default-hop flag ========================================================== 17:42:06 (1785879726) [ 3451.884211] Lustre: DEBUG MARKER: == sanity-lnet test 623: lnet_legacy2yaml: basic networks conversion ========================================================== 17:42:15 (1785879735) [ 3462.781457] Lustre: DEBUG MARKER: == sanity-lnet test 624: lnet_legacy2yaml: networks with interfaces ========================================================== 17:42:26 (1785879746) [ 3473.577952] Lustre: DEBUG MARKER: == sanity-lnet test 625: lnet_legacy2yaml: multiple networks ========================================================== 17:42:36 (1785879756) [ 3484.213669] Lustre: DEBUG MARKER: == sanity-lnet test 626: lnet_legacy2yaml: mixed networks, ip2nets, and routes ========================================================== 17:42:47 (1785879767) [ 3496.826756] Lustre: DEBUG MARKER: == sanity-lnet test 627: lnet_legacy2yaml: invalid networks format ========================================================== 17:43:00 (1785879780) [ 3508.759931] Lustre: DEBUG MARKER: == sanity-lnet test 628: lnet_legacy2yaml: missing delimiter error ========================================================== 17:43:11 (1785879791) [ 3520.637164] Lustre: DEBUG MARKER: == sanity-lnet test 629: lnet_legacy2yaml: multiple gateways error ========================================================== 17:43:24 (1785879804) [ 3530.205419] Lustre: DEBUG MARKER: == sanity-lnet test complete, duration 3249 sec ========== 17:43:33 (1785879813) [ 3530.964087] LNet: 144489:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3530.969919] LNetError: 144489:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3531.002146] LNet: Removed LNI 192.168.201.5@tcp [ 3531.682848] Key type .llcrypt unregistered [ 3531.688352] Key type ._llcrypt unregistered [ 3550.133449] Key type ._llcrypt registered [ 3550.141483] Key type .llcrypt registered [ 3550.922809] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 3550.976268] alg: No test for adler32 (adler32-zlib) [ 3552.488896] Lustre: Lustre: Build Version: 2.17.56_50_g55fd350 [ 3552.818275] LNet: Added LNI 192.168.201.5@tcp [8/256/0/180] [ 3554.599618] Key type lgssc registered [ 3556.748721] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3655.758676] Lustre: Mounted lustre-client - version 2.17.56_50_g55fd350 [ 3663.391828] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3674.679895] Lustre: Unmounted lustre-client [ 3711.275356] Key type lgssc unregistered [ 3711.546259] LNet: 147455:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3711.567229] LNetError: 147455:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3711.600250] LNet: Removed LNI 192.168.201.5@tcp [ 3713.008394] Key type .llcrypt unregistered [ 3713.013190] Key type ._llcrypt unregistered