[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 530740713 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54d0-0x000f54df] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829700K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002308] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004009] kvm-guest: setup PV IPIs [ 0.007304] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008016] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009007] pid_max: default: 32768 minimum: 301 [ 0.010130] LSM: Security Framework initializing [ 0.011044] Yama: becoming mindful. [ 0.012030] SELinux: Initializing. [ 0.013053] *** VALIDATE selinux *** [ 0.021029] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026461] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027131] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028105] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029096] *** VALIDATE tmpfs *** [ 0.030442] *** VALIDATE proc *** [ 0.031221] *** VALIDATE cgroup *** [ 0.032007] *** VALIDATE cgroup2 *** [ 0.033246] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034144] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036024] Spectre V2 : User space: Vulnerable [ 0.037005] Speculative Store Bypass: Vulnerable [ 0.040181] debug: unmapping init [mem 0xffffffffa3859000-0xffffffffa3860fff] [ 0.042983] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.043656] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.044021] ... version: 2 [ 0.045007] ... bit width: 48 [ 0.046008] ... generic registers: 4 [ 0.047065] ... value mask: 0000ffffffffffff [ 0.048007] ... max period: 00007fffffffffff [ 0.049006] ... fixed-purpose events: 3 [ 0.049760] ... event mask: 000000070000000f [ 0.050293] rcu: Hierarchical SRCU implementation. [ 0.052129] smp: Bringing up secondary CPUs ... [ 0.053356] x86: Booting SMP configuration: [ 0.054019] .... node #0, CPUs: #1 #2 #3 [ 0.057142] smp: Brought up 1 node, 4 CPUs [ 0.059010] smpboot: Max logical packages: 1 [ 0.060015] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.099202] node 0 deferred pages initialised in 36ms [ 0.102138] devtmpfs: initialized [ 0.103178] x86/mm: Memory block size: 128MB [ 0.105902] gcov: version magic: 0x41383552 [ 0.109244] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.112081] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.114284] pinctrl core: initialized pinctrl subsystem [ 0.117137] [ 0.117620] ************************************************************* [ 0.119008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.121009] ** ** [ 0.124010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.126008] ** ** [ 0.128009] ** This means that this kernel is built to expose internal ** [ 0.130009] ** IOMMU data structures, which may compromise security on ** [ 0.132008] ** your system. ** [ 0.134008] ** ** [ 0.135013] ** If you see this message and you are not debugging the ** [ 0.137008] ** kernel, report this immediately to your vendor! ** [ 0.139008] ** ** [ 0.140009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.142007] ************************************************************* [ 0.143753] NET: Registered protocol family 16 [ 0.145622] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.148043] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.150040] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.152543] cpuidle: using governor menu [ 0.153793] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.155389] PCI: Using configuration type 1 for base access [ 0.157113] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.165045] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.166052] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.170053] cryptd: max_cpu_qlen set to 1000 [ 0.172205] ACPI: Added _OSI(Module Device) [ 0.173010] ACPI: Added _OSI(Processor Device) [ 0.174010] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.176009] ACPI: Added _OSI(Processor Aggregator Device) [ 0.181815] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.186501] ACPI: Interpreter enabled [ 0.188090] ACPI: PM: (supports S0 S3 S4 S5) [ 0.189042] ACPI: Using IOAPIC for interrupt routing [ 0.190064] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.192323] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.200940] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.202031] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.204059] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.206110] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.209359] acpiphp: Slot [2] registered [ 0.211076] acpiphp: Slot [5] registered [ 0.212028] acpiphp: Slot [6] registered [ 0.212951] acpiphp: Slot [3] registered [ 0.213062] acpiphp: Slot [4] registered [ 0.214080] acpiphp: Slot [7] registered [ 0.216072] acpiphp: Slot [8] registered [ 0.217061] acpiphp: Slot [9] registered [ 0.218079] acpiphp: Slot [10] registered [ 0.220115] acpiphp: Slot [11] registered [ 0.221096] acpiphp: Slot [12] registered [ 0.222057] acpiphp: Slot [13] registered [ 0.223066] acpiphp: Slot [14] registered [ 0.225061] acpiphp: Slot [15] registered [ 0.226061] acpiphp: Slot [16] registered [ 0.227079] acpiphp: Slot [17] registered [ 0.228087] acpiphp: Slot [18] registered [ 0.230061] acpiphp: Slot [19] registered [ 0.231112] acpiphp: Slot [20] registered [ 0.232080] acpiphp: Slot [21] registered [ 0.233114] acpiphp: Slot [22] registered [ 0.234060] acpiphp: Slot [23] registered [ 0.235069] acpiphp: Slot [24] registered [ 0.237064] acpiphp: Slot [25] registered [ 0.238135] acpiphp: Slot [26] registered [ 0.239089] acpiphp: Slot [27] registered [ 0.240079] acpiphp: Slot [28] registered [ 0.242182] acpiphp: Slot [29] registered [ 0.243092] acpiphp: Slot [30] registered [ 0.244063] acpiphp: Slot [31] registered [ 0.246054] PCI host bridge to bus 0000:00 [ 0.247011] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.249012] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.251044] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.253014] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.254011] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.256015] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.257198] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.259829] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.261927] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.268012] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.271579] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.273013] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.275012] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.277009] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.278470] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.280596] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.282028] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.284895] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.288011] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.297016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.302010] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.306714] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.323015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.335016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.344015] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.353134] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.361014] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.366013] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.378014] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.388859] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.391394] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.393362] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.396349] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.398126] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.403082] iommu: Default domain type: Passthrough [ 0.404454] SCSI subsystem initialized [ 0.406154] ACPI: bus type USB registered [ 0.408109] usbcore: registered new interface driver usbfs [ 0.410094] usbcore: registered new interface driver hub [ 0.412078] usbcore: registered new device driver usb [ 0.413114] pps_core: LinuxPPS API ver. 1 registered [ 0.414008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.417048] PTP clock support registered [ 0.418124] EDAC MC: Ver: 3.0.0 [ 0.420124] PCI: Using ACPI for IRQ routing [ 0.421688] NetLabel: Initializing [ 0.423011] NetLabel: domain hash size = 128 [ 0.425010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.427079] NetLabel: unlabeled traffic allowed by default [ 0.429131] vgaarb: loaded [ 0.431298] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.433015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.441109] clocksource: Switched to clocksource kvm-clock [ 0.553167] VFS: Disk quotas dquot_6.6.0 [ 0.554782] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.557258] *** VALIDATE ramfs *** [ 0.558319] *** VALIDATE hugetlbfs *** [ 0.559791] pnp: PnP ACPI init [ 0.561800] pnp: PnP ACPI: found 6 devices [ 0.582237] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.585989] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.587947] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.589651] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.591436] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.593316] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.595496] NET: Registered protocol family 2 [ 0.597691] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.602693] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.605698] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.610361] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.613471] TCP: Hash tables configured (established 65536 bind 65536) [ 0.615507] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.618064] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.620565] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.623311] NET: Registered protocol family 1 [ 0.625879] RPC: Registered named UNIX socket transport module. [ 0.627923] RPC: Registered udp transport module. [ 0.629542] RPC: Registered tcp transport module. [ 0.631031] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.633286] NET: Registered protocol family 44 [ 0.635064] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.637269] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.639579] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.641437] PCI: CLS 0 bytes, default 64 [ 0.642712] Unpacking initramfs... [ 2.061093] debug: unmapping init [mem 0xffff92407cc64000-0xffff92407ffcffff] [ 2.064914] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.067168] software IO TLB: mapped [mem 0x00000000a2200000-0x00000000a6200000] (64MB) [ 2.068907] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.574145] Initialise system trusted keyrings [ 2.575492] Key type blacklist registered [ 2.577039] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.585237] zbud: loaded [ 2.587770] *** VALIDATE nfs *** [ 2.588619] *** VALIDATE nfs4 *** [ 2.589854] pstore: using deflate compression [ 2.593164] Platform Keyring initialized [ 2.709170] NET: Registered protocol family 38 [ 2.711250] Key type asymmetric registered [ 2.713915] Asymmetric key parser 'x509' registered [ 2.715955] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.718955] io scheduler mq-deadline registered [ 2.720598] io scheduler kyber registered [ 2.722508] io scheduler bfq registered [ 2.724392] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.727702] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.731154] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.736846] ACPI: Power Button [PWRF] [ 2.743699] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.751419] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.761442] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.789117] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.819804] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.824647] Non-volatile memory driver v1.3 [ 2.826455] Linux agpgart interface v0.103 [ 2.857796] virtio_blk virtio1: [vda] 139288 512-byte logical blocks (71.3 MB/68.0 MiB) [ 2.861968] vda: detected capacity change from 0 to 71315456 [ 2.874647] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.877649] vdb: detected capacity change from 0 to 1073741824 [ 2.883402] libphy: Fixed MDIO Bus: probed [ 2.890513] usbcore: registered new interface driver usbserial_generic [ 2.892489] usbserial: USB Serial support registered for generic [ 2.894240] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.897447] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.898737] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.901790] mousedev: PS/2 mouse device common for all mice [ 2.904389] rtc_cmos 00:05: RTC can wake from S4 [ 2.904413] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.908401] rtc_cmos 00:05: registered as rtc0 [ 2.911834] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.912583] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.918383] intel_pstate: CPU model not supported [ 2.918594] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.924690] hid: raw HID events driver (C) Jiri Kosina [ 2.927891] usbcore: registered new interface driver usbhid [ 2.930033] usbhid: USB HID core driver [ 2.931510] drop_monitor: Initializing network drop monitor service [ 2.934064] Initializing XFRM netlink socket [ 2.936179] NET: Registered protocol family 10 [ 2.939390] Segment Routing with IPv6 [ 2.940654] NET: Registered protocol family 17 [ 2.942626] mpls_gso: MPLS GSO support [ 2.947843] RAS: Correctable Errors collector initialized. [ 2.949376] AVX version of gcm_enc/dec engaged. [ 2.950503] AES CTR mode by8 optimization enabled [ 3.036696] sched_clock: Marking stable (3036668251, 0)->(3870483975, -833815724) [ 3.040306] registered taskstats version 1 [ 3.042423] Loading compiled-in X.509 certificates [ 3.044710] zswap: loaded using pool lzo/zbud [ 3.070549] Key type big_key registered [ 3.085170] Key type encrypted registered [ 3.086966] ima: No TPM chip found, activating TPM-bypass! [ 3.089407] ima: Allocated hash algorithm: sha1 [ 3.091324] ima: No architecture policies found [ 3.093283] evm: Initialising EVM extended attributes: [ 3.095377] evm: security.selinux [ 3.096640] evm: security.ima [ 3.097910] evm: security.capability [ 3.099458] evm: HMAC attrs: 0x1 [ 3.101986] rtc_cmos 00:05: setting system clock to 2026-06-17 10:44:00 UTC (1781693040) [ 3.109243] debug: unmapping init [mem 0xffffffffa4803000-0xffffffffa49fffff] [ 3.112663] debug: unmapping init [mem 0xffffffffa3582000-0xffffffffa3858fff] [ 3.125100] Write protecting the kernel read-only data: 28672k [ 3.128607] debug: unmapping init [mem 0xffffffffa1c03000-0xffffffffa1dfffff] [ 3.131410] debug: unmapping init [mem 0xffffffffa2514000-0xffffffffa25fffff] [ 3.171573] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.181220] systemd[1]: Detected virtualization kvm. [ 3.183290] systemd[1]: Detected architecture x86-64. [ 3.185394] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.211838] systemd[1]: No hostname configured. [ 3.213818] systemd[1]: Set hostname to . [ 3.215938] random: systemd: uninitialized urandom read (16 bytes read) [ 3.218520] systemd[1]: Initializing machine ID from random generator. [ 3.279660] random: ln: uninitialized urandom read (6 bytes read) [ 3.377696] random: systemd: uninitialized urandom read (16 bytes read) [ 3.380524] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.385985] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.390827] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Slices. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. [ OK ] Reached target Paths. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... Starting Journal Service... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.193369] device-mapper: uevent: version 1.0.3 [ 4.195529] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.944977] virtio_net virtio0 ens2: renamed from eth0 Starting dracut initqueue hook... [ 5.037321] scsi host0: ata_piix [ 5.132040] scsi host1: ata_piix [ 5.134633] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 5.136931] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 8.920779] dracut-initqueue[584]: RTNETLINK answers: File exists [ 9.620767] random: crng init done [ 9.622200] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.352525] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.661488] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.981672] SELinux: Disabled at runtime. [ 12.040022] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.048438] systemd[1]: Detected virtualization kvm. [ 12.049564] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.545107] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.548717] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.554129] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.560410] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.563106] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.570363] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.581224] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Kernel Debug File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. Mounting Huge Pages File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local Encrypted Volumes. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Apply Kernel Variables... [ OK ] Reached target rpc_pipefs.target. Starting Remount Root and Kernel File Systems... [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. Mounting POSIX Message Queue File System... [ OK ] Listening on initctl Compatibility Named Pipe. Starting udev Coldplug all Devices... Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Created slice system-getty.slice. [ 12.852705] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 13.275635] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.950552] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.018679] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.021597] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.070235] EDAC sbridge: Ver: 1.1.2 [ 15.614647] Key type dns_resolver registered [ 15.905398] NFS: Registering the id_resolver key type [ 15.907542] Key type id_resolver registered [ 15.909148] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg122-client login: [ 41.127391] libcfs: loading out-of-tree module taints kernel. [ 41.200395] Key type ._llcrypt registered [ 41.201393] Key type .llcrypt registered [ 41.405096] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 41.409220] alg: No test for adler32 (adler32-zlib) [ 42.393196] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 42.688876] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 44.304128] Key type lgssc registered [ 44.915979] Lustre: Echo OBD driver; http://www.lustre.org/ [ 91.771487] Lustre: Mounted lustre-client [ 94.285553] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 103.085134] Lustre: DEBUG MARKER: oleg122-client.virtnet: executing check_logdir /tmp/testlogs/ [ 104.725927] Lustre: DEBUG MARKER: oleg122-client.virtnet: executing yml_node [ 106.416887] Lustre: DEBUG MARKER: Client: 2.17.54.40 [ 107.559343] Lustre: DEBUG MARKER: MDS: 2.17.54.40 [ 108.706487] Lustre: DEBUG MARKER: OSS: 2.17.54.40 [ 109.407325] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Wed Jun 17 06:45:46 EDT 2026 [ 112.608285] Lustre: lustre-OST0000-osc-ffff9240d156f800: disconnect after 20s idle [ 115.394115] Lustre: Unmounted lustre-client [ 143.384866] Key type lgssc unregistered [ 143.519737] LNet: 8388:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 143.521974] LNetError: 8388:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 143.535363] LNet: Removed LNI 192.168.201.22@tcp [ 143.854155] Key type .llcrypt unregistered [ 143.855331] Key type ._llcrypt unregistered [ 155.441882] Key type ._llcrypt registered [ 155.443529] Key type .llcrypt registered [ 155.541531] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 155.549434] alg: No test for adler32 (adler32-zlib) [ 156.393991] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 156.438940] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 156.452935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show [ 158.508984] Lustre: DEBUG MARKER: excepting tests: 253 254 410 [ 158.573153] LNet: 9261:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 158.584177] LNetError: 9261:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 158.612800] LNet: Removed LNI 192.168.201.22@tcp [ 158.979649] Key type .llcrypt unregistered [ 158.981629] Key type ._llcrypt unregistered [ 161.961612] Key type ._llcrypt registered [ 161.962971] Key type .llcrypt registered [ 162.686388] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 06:46:39 (1781693199) [ 162.845853] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 162.851713] alg: No test for adler32 (adler32-zlib) [ 163.675508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 163.712308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 163.722794] LNet: 9679:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 163.734358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 163.769484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 163.781445] LNet: 9686:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 166.931367] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 06:46:43 (1781693203) [ 166.998327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 167.010552] LNet: 10260:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 167.028152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 167.067557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp [ 170.013352] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 06:46:46 (1781693206) [ 170.081839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 170.101190] LNet: 10845:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 170.123160] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 170.163142] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 2.2.2.2@o2ib [ 172.940217] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 06:46:49 (1781693209) [ 172.983565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 172.995173] LNet: 11425:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 173.007687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 173.046599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib [ 175.920166] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 06:46:52 (1781693212) [ 176.014163] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 176.040276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 176.077522] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp [ 176.262476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp [ 179.408860] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 06:46:56 (1781693216) [ 179.550674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 179.560476] LNet: 12603:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 179.563330] LNet: 12603:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 179.577431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 179.613453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 179.797439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 179.977724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib [ 183.065538] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 06:46:59 (1781693219) [ 183.103877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 183.122349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 183.166181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi,[6-12/4]@bxi3f [ 186.058215] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 06:47:02 (1781693222) [ 186.099367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 186.126573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 186.168152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp [ 186.182690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 186.385716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib [ 186.407829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 186.584627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp [ 186.597744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 186.766373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib [ 186.782164] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 186.980172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib [ 187.005718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 187.231272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni [ 187.253233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 187.445402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni [ 187.462342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 187.640796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi [ 187.663261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 187.870082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi [ 187.883593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 188.046387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f [ 188.064443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 188.238411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f --nid [8-12]@bxi3f [ 188.259546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 188.432711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi,[3-7]@bxi3f [ 188.451822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 188.646760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 188.665416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 191.571393] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 06:47:08 (1781693228) [ 191.657626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 191.668508] LNet: 14462:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 191.679117] LNet: 14462:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 191.698360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 191.735245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp [ 191.755560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp [ 194.871674] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 06:47:11 (1781693231) [ 194.916183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 194.943084] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 194.984665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 195.001365] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 198.118848] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 06:47:14 (1781693234) [ 198.177881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 198.194968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 198.239811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp [ 198.254911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp [ 200.925352] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 06:47:17 (1781693237) [ 200.988339] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 201.025834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 201.062977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 201.086514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 203.955844] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 06:47:20 (1781693240) [ 204.015326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 204.034060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 204.071288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp [ 204.090539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp [ 206.923412] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 06:47:23 (1781693243) [ 206.989679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 207.010025] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 207.046678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 207.064032] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 209.670114] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 06:47:26 (1781693246) [ 209.711841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 209.723575] LNet: 17972:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 209.727071] LNet: 17972:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 209.738783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 209.799605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 209.959535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 212.456719] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 06:47:29 (1781693249) [ 212.515454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 212.537092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 212.634908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 212.743572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 215.447559] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 06:47:32 (1781693252) [ 215.513030] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 215.534315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 215.582720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 215.671269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 218.356496] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 06:47:35 (1781693255) [ 218.408541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 218.434116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 218.656531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 218.773527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 221.840764] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 06:47:38 (1781693258) [ 221.877215] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 221.907450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 221.953192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 221.974100] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 224.909271] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 06:47:41 (1781693261) [ 224.967169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 224.987761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 225.021880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 225.041286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 228.081439] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 06:47:44 (1781693264) [ 228.138782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 228.159372] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 228.196679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 19@gni [ 231.284102] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 06:47:47 (1781693267) [ 231.354775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 231.377194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 231.415576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib [ 234.297574] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 06:47:51 (1781693271) [ 234.329582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 234.346880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 234.386913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni [ 234.541229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni [ 236.955828] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 06:47:53 (1781693273) [ 236.997283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 237.020446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 237.057767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni [ 237.075581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 22@gni --nid 26@gni [ 239.682532] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 06:47:56 (1781693276) [ 239.717187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 239.741376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 239.777907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni [ 239.798615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni [ 242.276372] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 06:47:59 (1781693279) [ 242.307818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 242.319455] LNet: 24577:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 242.323126] LNet: 24577:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 10 previous similar messages [ 242.335347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 242.371573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni [ 242.388150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp [ 245.033398] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 06:48:01 (1781693281) [ 245.111806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 245.144491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 245.178920] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 245.195710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 247.854482] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 06:48:04 (1781693284) [ 247.910982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 247.935933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 247.980258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim [ 248.011116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp [ 248.019331] LNetError: 25767:0:(peer.c:2175:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 248.042283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --force [ 248.065526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show --nid 1.1.1.1@tcp [ 250.627928] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 06:48:07 (1781693287) [ 250.722233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 250.751514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 250.799142] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.65xwNk9SRm [ 250.847916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.65xwNk9SRm [ 250.880581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.65xwNk9SRm [ 253.850029] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 06:48:10 (1781693290) [ 253.950743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 253.968744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 254.000795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp [ 254.014765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp [ 254.033148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer list [ 256.705059] Lustre: DEBUG MARKER: == sanity-lnet test 50: Enable/disable routing via yaml import ========================================================== 06:48:13 (1781693293) [ 256.736886] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 256.753423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 256.780747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 257.840053] LNet: Message forwarding will be enabled in 2s [ 260.896205] LNet: Message forwarding enabled [ 261.065964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 261.077720] LNet: Message forwarding will stop in 2s to 6s [ 265.279526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 265.791345] LNet: Message forwarding disabled [ 266.388725] LNet: Message forwarding will be enabled in 2s [ 268.780370] Lustre: DEBUG MARKER: == sanity-lnet test 51: Set numa range via yaml import === 06:48:25 (1781693305) [ 268.816392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 269.410259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 269.443730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-51.yaml [ 272.168802] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 06:48:28 (1781693308) [ 272.220883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 272.243483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 272.277677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid foobar [ 272.293129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid foobar [ 272.312697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@o2ib [ 272.329558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --nid 1.1.1.1@tcp [ 272.345937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --nid 1.1.1.1@tcp [ 272.361082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add [ 272.375148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 272.390897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp [ 272.409284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@foo [ 272.427445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@tcp [ 272.438895] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@o2ib [ 272.457738] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@gni [ 272.477816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni [ 272.504483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni [ 272.524118] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp [ 272.542728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp [ 272.556863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp [ 272.578480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 272.596064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp --gateway 1@gni [ 272.605995] LNetError: 28833:0:(router.c:624:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni [ 275.451531] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 06:48:32 (1781693312) [ 275.484415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 275.506380] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 275.548406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 278.269674] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 06:48:34 (1781693314) [ 279.134895] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_100 Need at least 2 remote nodes found oleg122-server [ 279.949151] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 06:48:36 (1781693316) [ 279.982277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 279.999821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 280.032209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 280.044493] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 280.803878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp1 --gateway 192.168.201.[23-27]@tcp [ 280.818190] LNet: 30547:0:(router.c:728:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 280.831487] LNetError: 30336:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.201.24@tcp is being used as a gateway but routing feature is not turned on [ 280.839502] LNetError: 30336:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) route to tcp1 through 192.168.201.24@tcp has gone from up to down [ 281.892671] LNetError: 30336:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.201.23@tcp is being used as a gateway but routing feature is not turned on [ 281.906166] LNetError: 30336:0:(router.c:393:lnet_router_discovery_ping_reply()) Skipped 7 previous similar messages [ 282.918265] LNetError: 30336:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.201.23@tcp is being used as a gateway but routing feature is not turned on [ 282.929052] LNetError: 30336:0:(router.c:393:lnet_router_discovery_ping_reply()) Skipped 3 previous similar messages [ 283.716748] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 06:48:40 (1781693320) [ 283.937481] LNetError: 30548:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) route to tcp1 through 192.168.201.26@tcp has gone from up to down [ 283.943242] LNetError: 30548:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) Skipped 3 previous similar messages [ 284.555271] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_102 Need at least 2 remote nodes found oleg122-server [ 284.965219] LNetError: 30336:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.201.24@tcp is being used as a gateway but routing feature is not turned on [ 284.972135] LNetError: 30336:0:(router.c:393:lnet_router_discovery_ping_reply()) Skipped 7 previous similar messages [ 285.332445] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 06:48:42 (1781693322) [ 285.376945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 285.984153] LNetError: 30335:0:(router.c:1266:lnet_check_routers()) Failed to discover router 192.168.201.23@tcp [ 287.457154] LNetError: 31475:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 287.470207] LNet: Removed LNI 192.168.201.22@tcp [ 287.486321] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 287.509757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 287.526547] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 287.796357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp103 --gateway 192.168.201.[23-28/2]@tcp [ 287.810531] LNet: 31599:0:(router.c:728:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 287.816081] LNet: 31599:0:(router.c:728:lnet_add_route()) Skipped 4 previous similar messages [ 287.825676] LNetError: 31486:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) route to tcp103 through 192.168.201.27@tcp has gone from up to down [ 287.827632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route del --net tcp103 --gateway 192.168.201.[23-28/2]@tcp [ 290.542163] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 06:48:47 (1781693327) [ 290.595809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 290.606125] LNetError: 32187:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 290.623956] LNet: Removed LNI 192.168.201.22@tcp [ 290.640397] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 290.688980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking -1 [ 290.723496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 290.740638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 290.770673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 290.800739] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 290.816348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 290.850920] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 0 [ 290.896092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 290.916668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 290.944101] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 290.984806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 291.003726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 291.029625] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 1 [ 291.071639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 291.088480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 291.113092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 291.149993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 291.182390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 291.218229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 2 [ 291.268550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 291.292974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 291.329824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 291.381682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 291.407730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 291.441457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 291.475562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 291.487703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 291.515108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 291.558489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 291.581117] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 291.609177] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 4 [ 291.632589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 291.653812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 291.688385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 294.528750] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 06:48:51 (1781693331) [ 295.395116] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_105 Need at least 2 remote nodes found oleg122-server [ 296.253695] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 06:48:52 (1781693332) [ 297.235994] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_106 Need at least 2 remote nodes found oleg122-server [ 298.032639] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 06:48:54 (1781693334) [ 298.390129] Key type .llcrypt unregistered [ 298.391578] Key type ._llcrypt unregistered [ 300.756023] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 300.828175] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 300.838553] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 300.842086] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 303.098130] Key type ._llcrypt registered [ 303.099175] Key type .llcrypt registered [ 303.177075] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 303.187937] alg: No test for adler32 (adler32-zlib) [ 304.001737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 304.043646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 304.056939] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 304.396787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 304.407064] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 305.413993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 305.424554] LNet: Removed LNI 10.1.2.3@tcp [ 305.485406] LNet: 34303:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 305.491567] LNetError: 34303:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 305.506046] LNet: Removed LNI 192.168.201.22@tcp [ 305.854635] Key type .llcrypt unregistered [ 305.857165] Key type ._llcrypt unregistered [ 311.311271] Key type ._llcrypt registered [ 311.313133] Key type .llcrypt registered [ 311.981374] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 06:49:08 (1781693348) [ 312.203587] Key type .llcrypt unregistered [ 312.206611] Key type ._llcrypt unregistered [ 314.832158] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 314.875144] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 314.878368] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 314.884342] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 317.222076] Key type ._llcrypt registered [ 317.223733] Key type .llcrypt registered [ 317.311313] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 317.318561] alg: No test for adler32 (adler32-zlib) [ 318.130189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 318.159696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 318.171794] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 318.501387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 318.511029] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 319.647423] LNet: 35795:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 319.653745] LNetError: 35795:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 319.658525] LNet: Removed LNI 192.168.201.22@tcp [ 319.667406] LNet: Removed LNI 10.1.2.3@tcp [ 320.014521] Key type .llcrypt unregistered [ 320.016064] Key type ._llcrypt unregistered [ 324.350868] Key type ._llcrypt registered [ 324.352681] Key type .llcrypt registered [ 324.820322] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 06:49:21 (1781693361) [ 324.979437] Key type .llcrypt unregistered [ 324.981080] Key type ._llcrypt unregistered [ 327.002609] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 327.074364] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 327.076351] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 327.078431] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 329.345569] Key type ._llcrypt registered [ 329.347100] Key type .llcrypt registered [ 329.432535] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 329.439572] alg: No test for adler32 (adler32-zlib) [ 330.258930] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 330.290735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 330.304756] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 331.224184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 331.236723] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 332.118077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 332.128834] LNet: Removed LNI 10.1.2.3@tcp [ 332.137760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg:0 [ 332.159442] LNet: Removed LNI 10.1.2.31@tcp [ 332.167313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 332.178399] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 332.471834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 332.481268] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 332.891152] LNet: 37713:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 332.895392] LNetError: 37713:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 332.898569] LNet: Removed LNI 10.1.2.3@tcp [ 332.906646] LNet: Removed LNI 10.1.2.31@tcp [ 333.218427] Key type .llcrypt unregistered [ 333.220760] Key type ._llcrypt unregistered [ 338.216556] Key type ._llcrypt registered [ 338.218592] Key type .llcrypt registered [ 338.839913] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 06:49:35 (1781693375) [ 338.978733] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 338.983916] alg: No test for adler32 (adler32-zlib) [ 339.806238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 339.838329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 339.850914] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 340.199667] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 340.210291] LNet: 38666:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 340.213608] LNetError: 38666:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 340.225362] LNet: Removed LNI 192.168.201.22@tcp [ 340.233466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 340.257833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --nid 192.168.201.22@tcp [ 340.270637] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 340.577484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --nid 192.168.201.22@tcp [ 340.598161] LNet: Removed LNI 192.168.201.22@tcp [ 340.908032] Key type .llcrypt unregistered [ 340.910498] Key type ._llcrypt unregistered [ 345.583513] Key type ._llcrypt registered [ 345.585027] Key type .llcrypt registered [ 346.224661] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 06:49:42 (1781693382) [ 347.072697] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_111 Need at least 2 remote nodes found oleg122-server [ 347.780726] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 06:49:44 (1781693384) [ 347.977385] Key type .llcrypt unregistered [ 347.978580] Key type ._llcrypt unregistered [ 350.204789] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 350.245841] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 350.248770] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 350.252485] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 352.506988] Key type ._llcrypt registered [ 352.508415] Key type .llcrypt registered [ 352.589697] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 352.596534] alg: No test for adler32 (adler32-zlib) [ 353.419735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 353.449335] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 353.772149] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180] [ 354.671845] LNet: 40634:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 354.676416] LNetError: 40634:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 354.680114] LNet: Removed LNI 192.168.201.22@tcp1 [ 354.686360] LNet: Removed LNI 10.1.2.3@tcp2 [ 354.971559] Key type .llcrypt unregistered [ 354.973140] Key type ._llcrypt unregistered [ 359.171678] Key type ._llcrypt registered [ 359.173349] Key type .llcrypt registered [ 359.790439] Lustre: DEBUG MARKER: == sanity-lnet test 150: Check import of ip2nets yaml sequence ========================================================== 06:49:56 (1781693396) [ 359.951574] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 359.959108] alg: No test for adler32 (adler32-zlib) [ 360.776354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 360.802639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 360.817638] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 361.183455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 361.193771] LNet: 41601:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 361.197228] LNetError: 41601:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 361.205407] LNet: Removed LNI 192.168.201.22@tcp [ 361.215447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 361.245247] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet.yaml [ 361.256271] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 363.964195] Lustre: DEBUG MARKER: == sanity-lnet test 151: Check correct application of multiple ip2nets rules ========================================================== 06:50:00 (1781693400) [ 364.017445] LNet: 42186:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 364.021091] LNetError: 42186:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 364.028613] LNet: Removed LNI 192.168.201.22@tcp [ 364.366473] Key type .llcrypt unregistered [ 364.368312] Key type ._llcrypt unregistered [ 366.745115] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 366.785132] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 366.786497] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 366.789019] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 369.040193] Key type ._llcrypt registered [ 369.041877] Key type .llcrypt registered [ 369.110387] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 369.117287] alg: No test for adler32 (adler32-zlib) [ 369.927347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 369.952051] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 369.965516] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 370.289529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 370.300275] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 371.260119] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 371.270607] LNet: 42936:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 371.274086] LNetError: 42936:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 371.277790] LNet: Removed LNI 192.168.201.22@tcp [ 371.286534] LNet: Removed LNI 10.1.2.3@tcp [ 371.298413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 371.330423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 371.346115] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 371.349976] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 371.538135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 371.547539] LNet: Removed LNI 192.168.201.22@tcp [ 371.558461] LNet: Removed LNI 10.1.2.3@tcp [ 371.568558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 371.602393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 371.615503] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 371.618692] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 371.806000] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 371.817520] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 372.297122] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 372.313631] LNet: 43125:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 372.317128] LNet: 43125:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 372.320316] LNetError: 43125:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 372.324166] LNetError: 43125:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 372.329660] LNet: Removed LNI 192.168.201.22@tcp [ 372.332320] LNet: Removed LNI 10.1.2.3@tcp [ 372.340567] LNet: Removed LNI 192.168.201.22@tcp2 [ 372.351882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 372.382692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 372.394883] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 372.396701] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 372.399310] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 375.256899] Lustre: DEBUG MARKER: == sanity-lnet test 152: Check import of ip2nets with wildcard IP patterns ========================================================== 06:50:12 (1781693412) [ 375.345773] LNet: 43721:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 375.348812] LNetError: 43721:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 375.351808] LNetError: 43721:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 375.356338] LNet: Removed LNI 192.168.201.22@tcp [ 375.357792] LNet: Removed LNI 10.1.2.3@tcp [ 375.363658] LNet: Removed LNI 192.168.201.22@tcp2 [ 375.628053] Key type .llcrypt unregistered [ 375.629115] Key type ._llcrypt unregistered [ 377.849875] Key type ._llcrypt registered [ 377.850859] Key type .llcrypt registered [ 377.913400] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 377.919473] alg: No test for adler32 (adler32-zlib) [ 378.746192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 378.780064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 378.797095] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 379.157370] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 379.168669] LNet: 44130:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 379.171807] LNetError: 44130:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 379.180634] LNet: Removed LNI 192.168.201.22@tcp [ 379.196066] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 379.231571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 379.242776] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 379.431786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 379.451602] LNet: Removed LNI 192.168.201.22@tcp [ 379.463189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 379.489992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 379.503211] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 379.681366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 379.691198] LNet: 44190:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 379.693990] LNet: 44190:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 379.697320] LNetError: 44190:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 379.701261] LNetError: 44190:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 379.710899] LNet: Removed LNI 192.168.201.22@tcp [ 379.720186] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 379.754435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 379.767125] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 379.959465] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 380.029578] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 380.031710] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 380.033945] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 382.220408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 382.230728] LNet: 44264:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 382.234225] LNetError: 44264:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 382.243211] LNet: Removed LNI 192.168.201.22@tcp [ 382.256156] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 382.286446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 382.301115] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 384.677354] Lustre: DEBUG MARKER: == sanity-lnet test 153: Check import of ip2nets with IP address ranges ========================================================== 06:50:21 (1781693421) [ 384.727970] LNet: 44855:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 384.731386] LNetError: 44855:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 384.739466] LNet: Removed LNI 192.168.201.22@tcp [ 385.048924] Key type .llcrypt unregistered [ 385.050738] Key type ._llcrypt unregistered [ 387.672757] Key type ._llcrypt registered [ 387.674415] Key type .llcrypt registered [ 387.772780] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 387.780849] alg: No test for adler32 (adler32-zlib) [ 388.601980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 388.623915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 388.635076] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 389.017583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 389.032661] LNet: 45258:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 389.036926] LNetError: 45258:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 389.049582] LNet: Removed LNI 192.168.201.22@tcp [ 389.061660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 389.091571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-153.yaml [ 389.102530] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 392.274654] Lustre: DEBUG MARKER: == sanity-lnet test 154: Check import of ip2nets with stepped IP address ranges ========================================================== 06:50:29 (1781693429) [ 392.316389] LNet: 45844:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 392.322799] LNetError: 45844:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 392.330784] LNet: Removed LNI 192.168.201.22@tcp [ 392.597619] Key type .llcrypt unregistered [ 392.599607] Key type ._llcrypt unregistered [ 395.213921] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 395.288647] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 395.290893] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 395.292959] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 397.546489] Key type ._llcrypt registered [ 397.548961] Key type .llcrypt registered [ 397.645301] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 397.654546] alg: No test for adler32 (adler32-zlib) [ 398.491077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 398.525475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 398.538379] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 398.913286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 398.925851] LNet: 46299:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 398.930828] LNetError: 46299:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 398.944297] LNet: Removed LNI 192.168.201.22@tcp [ 398.955606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 398.995166] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-154.yaml [ 399.015537] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 402.060448] Lustre: DEBUG MARKER: == sanity-lnet test 155: Check ip2nets import failure with no matching rules ========================================================== 06:50:38 (1781693438) [ 402.099317] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 402.110572] LNet: 46889:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 402.115810] LNetError: 46889:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 402.129889] LNet: Removed LNI 192.168.201.22@tcp [ 402.142374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 402.193161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-155.yaml [ 404.623397] Lustre: DEBUG MARKER: == sanity-lnet test 156: Check import of ip2nets with comma-separated IP ranges ========================================================== 06:50:41 (1781693441) [ 404.663929] LNet: 47470:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 404.920476] Key type .llcrypt unregistered [ 404.922189] Key type ._llcrypt unregistered [ 407.083252] Key type ._llcrypt registered [ 407.084418] Key type .llcrypt registered [ 407.149626] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 407.155530] alg: No test for adler32 (adler32-zlib) [ 407.990430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 408.024606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 408.047582] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 408.456932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 408.468132] LNet: 47874:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 408.471382] LNetError: 47874:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 408.482411] LNet: Removed LNI 192.168.201.22@tcp [ 408.490883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 408.528993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-156.yaml [ 408.547480] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 411.341528] Lustre: DEBUG MARKER: == sanity-lnet test 157: Check import failure with malformed ip2nets YAML ========================================================== 06:50:48 (1781693448) [ 411.407240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 411.423292] LNet: 48463:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 411.426274] LNetError: 48463:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 411.436897] LNet: Removed LNI 192.168.201.22@tcp [ 411.445136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 411.473526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-157.yaml [ 414.084091] Lustre: DEBUG MARKER: == sanity-lnet test 158: Check import of ip2nets with single IPv6 address ========================================================== 06:50:50 (1781693450) [ 414.750178] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_158 Interface has no IPv6 [ 415.395961] Lustre: DEBUG MARKER: == sanity-lnet test 159: Check import of ip2nets with IPv6 /128 CIDR notation ========================================================== 06:50:52 (1781693452) [ 415.975066] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_159 Interface has no IPv6 [ 416.621765] Lustre: DEBUG MARKER: == sanity-lnet test 160: Check correct application of multiple IPv6 ip2nets rules ========================================================== 06:50:53 (1781693453) [ 417.312246] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_160 Interface has no IPv6 [ 418.098568] Lustre: DEBUG MARKER: == sanity-lnet test 161: Check import of ip2nets with various IPv6 prefix lengths ========================================================== 06:50:54 (1781693454) [ 418.807228] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_161 Interface has no IPv6 [ 419.581632] Lustre: DEBUG MARKER: == sanity-lnet test 162: Check ip2nets import failure with non-matching IPv6 ========================================================== 06:50:56 (1781693456) [ 420.259377] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_162 Interface has no IPv6 [ 420.981244] Lustre: DEBUG MARKER: == sanity-lnet test 170: Check CPTs and tunables parsing when importing ========================================================== 06:50:57 (1781693457) [ 421.033277] LNet: 50508:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 421.320522] Key type .llcrypt unregistered [ 421.322313] Key type ._llcrypt unregistered [ 423.597982] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 423.627679] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 423.630221] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 423.632685] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 425.895811] Key type ._llcrypt registered [ 425.897253] Key type .llcrypt registered [ 425.975231] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 425.981854] alg: No test for adler32 (adler32-zlib) [ 426.806633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 426.843207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 426.856305] LNet: Added LNI 192.168.201.22@tcp [8/512/0/180] [ 426.859118] LNet: Added LNI 10.1.2.3@tcp [8/512/0/180] [ 426.860912] LNet: 50835:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 426.899867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 426.908811] LNet: 50854:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 426.912988] LNetError: 50854:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 426.915189] LNet: Removed LNI 192.168.201.22@tcp [ 426.916333] LNetError: 50854:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 426.924538] LNet: Removed LNI 10.1.2.3@tcp [ 426.933772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 426.963736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 426.974932] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 427.005770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 427.015175] LNet: 50878:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 427.017805] LNetError: 50878:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 427.024374] LNet: Removed LNI 192.168.201.22@tcp [ 427.033766] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 427.062495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 427.072383] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 427.074808] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 427.076858] LNet: 50886:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 427.119940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 427.131167] LNet: 50905:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 427.133844] LNetError: 50905:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 427.136442] LNet: Removed LNI 192.168.201.22@tcp [ 427.137796] LNetError: 50905:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 427.150132] LNet: Removed LNI 10.1.2.3@tcp [ 427.160884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 427.203149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 427.216377] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 427.219296] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 427.220685] LNet: 50913:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 427.292257] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 427.302463] LNet: 50939:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 427.306108] LNetError: 50939:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 427.310534] LNet: Removed LNI 192.168.201.22@tcp [ 427.312761] LNetError: 50939:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 427.322338] LNet: Removed LNI 10.1.2.3@tcp [ 427.330735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 427.357395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 427.373319] LNet: Added LNI 192.168.201.22@tcp [8/256/0/182] [ 427.376135] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 427.416285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 427.424446] LNet: 50967:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 427.426419] LNetError: 50967:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 427.428918] LNet: Removed LNI 192.168.201.22@tcp [ 427.430286] LNetError: 50967:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 427.436411] LNet: Removed LNI 10.1.2.3@tcp [ 427.445908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 427.473307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 427.482877] LNet: Added LNI 192.168.201.22@tcp [8/256/0/182] [ 427.485971] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 427.487711] LNet: 50975:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 427.521146] LNet: Removed LNI 192.168.201.22@tcp [ 427.529363] LNet: Removed LNI 10.1.2.3@tcp [ 427.782054] Key type .llcrypt unregistered [ 427.783502] Key type ._llcrypt unregistered [ 431.822759] Key type ._llcrypt registered [ 431.824451] Key type .llcrypt registered [ 432.335468] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 06:51:09 (1781693469) [ 432.464913] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 432.470545] alg: No test for adler32 (adler32-zlib) [ 433.286394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 433.307076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 433.320265] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 433.617400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --ip2net tcp 192.168.201.22 [ 433.625988] LNet: 51965:0:(config.c:1627:lnet_parse_ip2nets()) Kernel parsing of ip2nets is deprecated. Consider converting to LNet YAML configuration. [ 433.630953] LNet: Removed LNI 192.168.201.22@tcp [ 435.805593] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 06:51:12 (1781693472) [ 435.840689] LNet: 52526:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 435.843156] LNetError: 52526:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 435.853622] LNet: Removed LNI 192.168.201.22@tcp [ 436.095350] Key type .llcrypt unregistered [ 436.096268] Key type ._llcrypt unregistered [ 438.080577] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 438.181412] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 440.462938] Key type ._llcrypt registered [ 440.464035] Key type .llcrypt registered [ 440.517135] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 440.523352] alg: No test for adler32 (adler32-zlib) [ 441.508376] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 443.864792] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 06:51:20 (1781693480) [ 443.899654] LNet: 53440:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 443.902883] LNetError: 53440:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 443.913732] LNet: Removed LNI 10.1.2.3@tcp [ 444.148377] Key type .llcrypt unregistered [ 444.149539] Key type ._llcrypt unregistered [ 446.040543] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 446.135392] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 446.137179] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 446.139643] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 448.393664] Key type ._llcrypt registered [ 448.395052] Key type .llcrypt registered [ 448.453847] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 448.460283] alg: No test for adler32 (adler32-zlib) [ 449.277303] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 451.241724] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 06:51:28 (1781693488) [ 451.713659] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_202 kernel ip2nets deprecated in 2.17.53 [ 452.250901] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 06:51:29 (1781693489) [ 452.281460] LNet: 54621:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 452.284676] LNetError: 54621:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 452.292396] LNet: Removed LNI 10.1.2.3@tcp [ 452.523386] Key type .llcrypt unregistered [ 452.524474] Key type ._llcrypt unregistered [ 454.381973] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 454.475761] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 454.477442] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 454.479528] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 456.746263] Key type ._llcrypt registered [ 456.747548] Key type .llcrypt registered [ 456.810136] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 456.815897] alg: No test for adler32 (adler32-zlib) [ 457.615389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 457.645121] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 460.306602] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 06:51:37 (1781693497) [ 460.437189] LNet: 55855:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 460.439645] LNetError: 55855:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 460.450595] LNet: Removed LNI 10.1.2.3@tcp [ 460.657453] Key type .llcrypt unregistered [ 460.658377] Key type ._llcrypt unregistered [ 462.651905] Key type ._llcrypt registered [ 462.653109] Key type .llcrypt registered [ 462.726345] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 462.733885] alg: No test for adler32 (adler32-zlib) [ 463.542686] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 463.552895] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 466.497091] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 466.735682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_interrupt -r 1 [ 466.751975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_interrupt -r 1 [ 466.767597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 466.777145] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 466.857415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 466.874452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 466.889874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 466.904432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 466.917732] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 466.932529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 466.947107] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 466.962126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 466.971227] LNet: 56291:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 466.979076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 466.994134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 467.007327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 467.020653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 467.034955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 467.049468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 467.063321] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 467.283471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_dropped -r 1 [ 467.298520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_dropped -r 1 [ 467.314651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 467.323586] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 467.404681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 467.417949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 467.432312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 467.446874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 467.460907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 467.474762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 467.487974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 467.500728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 467.508432] LNet: 56400:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 467.517253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 467.530339] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 467.542864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 467.555732] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 467.569803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 467.583552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 467.597666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 467.795833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_aborted -r 1 [ 467.812798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_aborted -r 1 [ 467.827794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 467.912549] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 467.925656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 467.940280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 467.953410] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 467.968250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 467.981341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 467.998072] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 468.013695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 468.029600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 468.042961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 468.055195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 468.068364] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 468.080902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 468.094821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 468.109117] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 468.307582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_no_route -r 1 [ 468.323149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_no_route -r 1 [ 468.342846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 468.352188] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 468.355771] LNet: Skipped 1 previous similar message [ 468.431914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 468.445808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 468.458331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 468.470390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 468.482377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 468.493891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 468.507287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 468.519645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 468.527590] LNet: 56618:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 468.530424] LNet: 56618:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 468.537937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 468.552031] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 468.565387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 468.578600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 468.592635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 468.606382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 468.625305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 468.837737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_timeout -r 1 [ 468.858187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_timeout -r 1 [ 468.877801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 468.976800] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 468.990704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 469.006777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 469.020451] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 469.037077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 469.050585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 469.065292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 469.082126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 469.097093] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 469.111946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 469.126753] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 469.141918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 469.156683] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 469.170959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 469.185857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 469.385886] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_error -r 1 [ 469.403944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_error -r 1 [ 469.419680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 469.505517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 469.519966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 469.534880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 469.551609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 469.565355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 469.581176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 469.596218] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 469.610807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 469.623858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 469.639405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 469.652184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 469.666945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 469.688560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 469.703255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 469.719493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 469.822860] LNet: 56877:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 469.825388] LNetError: 56877:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 469.834864] LNet: Removed LNI 192.168.201.22@tcp [ 469.991665] Key type .llcrypt unregistered [ 469.993501] Key type ._llcrypt unregistered [ 475.724864] Key type ._llcrypt registered [ 475.726397] Key type .llcrypt registered [ 476.264781] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 06:51:53 (1781693513) [ 476.563306] Key type .llcrypt unregistered [ 476.565062] Key type ._llcrypt unregistered [ 478.698965] Key type ._llcrypt registered [ 478.700365] Key type .llcrypt registered [ 478.767386] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 478.774505] alg: No test for adler32 (adler32-zlib) [ 479.578675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 479.590136] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 482.512859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 486.004660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 486.023341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 486.033948] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 486.608228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_interrupt -r 1 [ 486.627265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e local_interrupt -r 1 [ 486.649397] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_interrupt -r 1 [ 486.669261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e local_interrupt -r 1 [ 486.691458] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e local_interrupt -r 1 [ 486.709798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e local_interrupt -r 1 [ 486.726284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e local_interrupt -r 1 [ 486.747089] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e local_interrupt -r 1 [ 486.762150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 486.771255] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 486.773694] LNet: 1 local NIs in recovery (showing 1): 192.168.201.22@tcp [ 486.849731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 486.862967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 486.876097] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 486.889181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 486.905022] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 486.919820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 486.935763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 486.950567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 486.959509] LNet: 58350:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 486.969546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 486.982366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 486.996263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 487.010531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 487.023933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 487.037794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 487.051839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 487.249817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_dropped -r 1 [ 487.268284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e local_dropped -r 1 [ 487.285092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_dropped -r 1 [ 487.300687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e local_dropped -r 1 [ 487.318472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e local_dropped -r 1 [ 487.335419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e local_dropped -r 1 [ 487.357872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e local_dropped -r 1 [ 487.375644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e local_dropped -r 1 [ 487.391077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 487.400163] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 487.402523] LNet: Skipped 2 previous similar messages [ 487.482070] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 487.496314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 487.509402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 487.521136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 487.535430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 487.549173] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 487.562224] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 487.575434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 487.585377] LNet: 58471:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 487.593884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 487.607956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 487.619119] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 487.634399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 487.647573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 487.661576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 487.676349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 487.868911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_aborted -r 1 [ 487.884939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e local_aborted -r 1 [ 487.900319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_aborted -r 1 [ 487.917697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e local_aborted -r 1 [ 487.934543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e local_aborted -r 1 [ 487.951856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e local_aborted -r 1 [ 487.968857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e local_aborted -r 1 [ 487.985685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e local_aborted -r 1 [ 488.002039] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 488.084973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 488.096088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 488.108855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 488.122411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 488.136958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 488.150168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 488.163273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 488.178498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 488.192529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 488.204343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 488.217342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 488.230801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 488.243166] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 488.254953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 488.268574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 488.448319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_no_route -r 1 [ 488.466152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e local_no_route -r 1 [ 488.482960] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_no_route -r 1 [ 488.496978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e local_no_route -r 1 [ 488.512699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e local_no_route -r 1 [ 488.525727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e local_no_route -r 1 [ 488.541915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e local_no_route -r 1 [ 488.558306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e local_no_route -r 1 [ 488.576074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 488.585761] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 488.589160] LNet: Skipped 7 previous similar messages [ 488.677141] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 488.691037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 488.704472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 488.718897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 488.734622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 488.750415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 488.762424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 488.774232] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 488.781805] LNet: 58713:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 488.785521] LNet: 58713:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 488.793322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 488.804820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 488.817697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 488.831051] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 488.843890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 488.859813] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 488.876266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 489.059461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_timeout -r 1 [ 489.074616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e local_timeout -r 1 [ 489.089590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_timeout -r 1 [ 489.105606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e local_timeout -r 1 [ 489.121412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e local_timeout -r 1 [ 489.135924] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e local_timeout -r 1 [ 489.160389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e local_timeout -r 1 [ 489.179092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e local_timeout -r 1 [ 489.198209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 489.282618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 489.296179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 489.309853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 489.323996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 489.337406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 489.353628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 489.369257] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 489.384532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 489.400090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 489.413126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 489.425558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 489.439178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 489.452189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 489.465557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 489.482866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 489.710058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_error -r 1 [ 489.726452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e local_error -r 1 [ 489.741339] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_error -r 1 [ 489.757374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e local_error -r 1 [ 489.773985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e local_error -r 1 [ 489.792041] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e local_error -r 1 [ 489.810584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e local_error -r 1 [ 489.828713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e local_error -r 1 [ 489.844465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 489.947709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 489.963803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 489.979881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 489.996807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 490.012033] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 490.035330] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 490.050287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 490.064864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 490.082923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 490.105909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 490.122169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 490.138357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 490.155629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 490.172099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 490.188091] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 492.459501] LNet: 59031:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 492.462229] LNetError: 59031:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 492.465380] LNet: Removed LNI 192.168.201.22@tcp [ 493.478352] LNet: Removed LNI 192.168.201.22@tcp1 [ 493.623470] Key type .llcrypt unregistered [ 493.624870] Key type ._llcrypt unregistered [ 499.261023] Key type ._llcrypt registered [ 499.262705] Key type .llcrypt registered [ 499.775816] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 06:52:16 (1781693536) [ 500.116477] Key type .llcrypt unregistered [ 500.117777] Key type ._llcrypt unregistered [ 502.284449] Key type ._llcrypt registered [ 502.285515] Key type .llcrypt registered [ 502.369652] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 502.378291] alg: No test for adler32 (adler32-zlib) [ 503.188095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 503.199102] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 506.213649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 506.441704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e remote_dropped -r 1 [ 506.452924] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e remote_dropped -r 1 [ 506.465135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 506.472086] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 506.546419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 506.557395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 506.566983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 506.578154] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 506.590077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 506.602703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 506.614457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 506.627334] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 506.635799] LNet: 60277:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 506.644418] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 506.659049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 506.670558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 506.682359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 506.695737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 506.708735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 506.724446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 506.896676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e remote_error -r 1 [ 506.911152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e remote_error -r 1 [ 506.924430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 507.007545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 507.020214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 507.033897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 507.044734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 507.057900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 507.073515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 507.088816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 507.101314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 507.113882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 507.124685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 507.136988] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 507.150481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 507.161659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 507.174708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 507.187228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 507.346464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e remote_timeout -r 1 [ 507.361279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e remote_timeout -r 1 [ 507.374316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 507.382914] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 507.385166] LNet: Skipped 1 previous similar message [ 507.463664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 507.480179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 507.493167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 507.505677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 507.518305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 507.530263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 507.543784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 507.555558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 507.562226] LNet: 60495:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 507.565031] LNet: 60495:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 507.572554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 507.583911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 507.595405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 507.607075] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 507.617660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 507.626837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 507.639922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 507.727968] LNet: 60536:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 507.731298] LNetError: 60536:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 507.741327] LNet: Removed LNI 192.168.201.22@tcp [ 507.874332] Key type .llcrypt unregistered [ 507.875565] Key type ._llcrypt unregistered [ 513.396125] Key type ._llcrypt registered [ 513.397393] Key type .llcrypt registered [ 513.909650] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 06:52:30 (1781693550) [ 514.203287] Key type .llcrypt unregistered [ 514.204968] Key type ._llcrypt unregistered [ 516.245973] Key type ._llcrypt registered [ 516.247470] Key type .llcrypt registered [ 516.310253] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 516.316258] alg: No test for adler32 (adler32-zlib) [ 517.118472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 517.128592] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 519.947147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 522.504360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 522.516553] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 522.524430] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 522.966528] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e remote_dropped -r 1 [ 522.980860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e remote_dropped -r 1 [ 522.995105] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e remote_dropped -r 1 [ 523.011520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e remote_dropped -r 1 [ 523.026865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e remote_dropped -r 1 [ 523.042176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e remote_dropped -r 1 [ 523.059367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e remote_dropped -r 1 [ 523.076542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e remote_dropped -r 1 [ 523.093940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 523.104214] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 523.107590] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.122@tcp [ 523.181266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 523.192350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 523.203641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 523.216940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 523.231252] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 523.244750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 523.257809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 523.270692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 523.279698] LNet: 61997:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 523.288555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 523.299808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 523.311386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 523.321772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 523.332378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 523.344403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 523.356787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 523.371937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 523.543729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e remote_error -r 1 [ 523.557959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e remote_error -r 1 [ 523.572504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e remote_error -r 1 [ 523.586345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e remote_error -r 1 [ 523.598732] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e remote_error -r 1 [ 523.611496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e remote_error -r 1 [ 523.624179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e remote_error -r 1 [ 523.638589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e remote_error -r 1 [ 523.654947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 523.664242] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 523.666872] LNet: Skipped 2 previous similar messages [ 523.737585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 523.748302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 523.759972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 523.771184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 523.783436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 523.797139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 523.808281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 523.819539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 523.827577] LNet: 62120:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 523.835132] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 523.847254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 523.861872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 523.874244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 523.886317] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 523.899673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 523.911958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 523.925480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 524.099823] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e remote_timeout -r 1 [ 524.117634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e remote_timeout -r 1 [ 524.132589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e remote_timeout -r 1 [ 524.146921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e remote_timeout -r 1 [ 524.162342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e remote_timeout -r 1 [ 524.176473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e remote_timeout -r 1 [ 524.194037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e remote_timeout -r 1 [ 524.210827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e remote_timeout -r 1 [ 524.226638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.122@tcp [ 524.304554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 524.318373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 524.331086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 524.342337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 524.355865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 524.368649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 524.383460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 524.397907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 524.412292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 524.426022] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 524.439436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 524.452314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 524.466629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 524.479712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 524.493135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 524.506883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 526.654806] LNet: 62321:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 526.657592] LNetError: 62321:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 526.660642] LNet: Removed LNI 192.168.201.22@tcp [ 526.671316] LNet: Removed LNI 192.168.201.22@tcp1 [ 526.822451] Key type .llcrypt unregistered [ 526.823669] Key type ._llcrypt unregistered [ 532.427226] Key type ._llcrypt registered [ 532.428731] Key type .llcrypt registered [ 532.931224] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 06:52:49 (1781693569) [ 533.484487] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_208 kernel ip2nets deprecated in 2.17.53 [ 534.043037] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 06:52:50 (1781693570) [ 534.323264] Key type .llcrypt unregistered [ 534.324221] Key type ._llcrypt unregistered [ 536.319854] Key type ._llcrypt registered [ 536.321384] Key type .llcrypt registered [ 536.403695] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 536.412056] alg: No test for adler32 (adler32-zlib) [ 537.219613] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 537.229992] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 540.285153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 540.510750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e network_timeout -r 1 [ 540.525453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e network_timeout -r 1 [ 540.540067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 540.547732] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 540.619598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 540.632377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 540.646488] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 540.662811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 540.677106] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 540.690753] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 540.704375] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 540.719496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 540.728215] LNet: 63860:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 540.736799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 540.749321] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 540.763401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 540.776163] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 540.790207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 540.805554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 540.909047] LNet: 63899:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 540.912904] LNetError: 63899:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 540.922331] LNet: Removed LNI 192.168.201.22@tcp [ 541.089684] Key type .llcrypt unregistered [ 541.092195] Key type ._llcrypt unregistered [ 546.986103] Key type ._llcrypt registered [ 546.987670] Key type .llcrypt registered [ 547.054890] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 547.060827] alg: No test for adler32 (adler32-zlib) [ 547.866748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 547.877740] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 550.758883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 553.578421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 553.590367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 553.598191] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 554.134550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e network_timeout -r 1 [ 554.150467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e network_timeout -r 1 [ 554.166681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e network_timeout -r 1 [ 554.184098] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e network_timeout -r 1 [ 554.200738] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e network_timeout -r 1 [ 554.216788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e network_timeout -r 1 [ 554.234900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e network_timeout -r 1 [ 554.252863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e network_timeout -r 1 [ 554.267870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 554.277868] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 554.360277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 554.373856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 554.387556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 554.400966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 554.412868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 554.424645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 554.436710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 554.451834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 554.460168] LNet: 64774:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 554.468656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 554.484896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 554.498514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 554.514431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 554.531867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 554.546703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 556.748346] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 556.853570] LNet: 64847:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 556.855958] LNetError: 64847:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 556.859877] LNet: Removed LNI 192.168.201.22@tcp [ 556.867630] LNet: Removed LNI 192.168.201.22@tcp1 [ 557.023620] Key type .llcrypt unregistered [ 557.024847] Key type ._llcrypt unregistered [ 562.406608] Key type ._llcrypt registered [ 562.408127] Key type .llcrypt registered [ 562.870394] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 06:53:19 (1781693599) [ 563.009305] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 563.015196] alg: No test for adler32 (adler32-zlib) [ 563.820074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 563.843415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 563.854617] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 564.138591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 564.148510] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 564.472089] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 564.499751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 564.521336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 564.537583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 564.553720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 564.567856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 564.583533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.22@tcp [ 564.592793] LNetError: 65944:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.201.22@tcp to recovery [ 564.832233] LNet: 1 local NIs in recovery (showing 1): 192.168.201.22@tcp [ 564.832267] LNet: There was an unexpected network error while writing to 192.168.201.22: rc = -22 [ 566.880344] LNet: There was an unexpected network error while writing to 192.168.201.22: rc = -22 [ 567.750590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 570.976324] LNet: There was an unexpected network error while writing to 192.168.201.22: rc = -22 [ 571.943724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 571.957638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 571.983058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 571.992579] LNet: 66061:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 571.996522] LNetError: 66061:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 571.999811] LNet: Removed LNI 192.168.201.22@tcp [ 572.005357] LNet: Removed LNI 192.168.201.22@tcp1 [ 572.013507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 572.037063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 572.047933] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 572.321750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 572.330841] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 572.634788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 572.648148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 572.671346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 572.689704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 572.704747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 572.719480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 572.734371] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 572.750137] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.22@tcp [ 572.760422] LNetError: 66333:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.201.22@tcp to recovery [ 573.088300] LNet: There was an unexpected network error while writing to 192.168.201.22: rc = -22 [ 575.930485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 579.232378] LNet: There was an unexpected network error while writing to 192.168.201.22: rc = -22 [ 579.235295] LNet: Skipped 1 previous similar message [ 584.260348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 584.274416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 584.290133] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 586.567704] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 06:53:43 (1781693623) [ 586.593087] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 586.603395] LNet: 67053:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 586.605951] LNetError: 67053:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 586.608451] LNetError: 67053:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 586.611546] LNet: Removed LNI 192.168.201.22@tcp [ 586.616503] LNet: Removed LNI 192.168.201.22@tcp1 [ 586.627343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 586.649464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 586.658985] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 586.914987] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 586.925583] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 587.221865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 587.246901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 587.260969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 587.279870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 587.296504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 587.312722] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 587.329296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.22@tcp --health 0 [ 587.369468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 587.680105] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.22@tcp [ 588.704317] LNet: There was an unexpected network error while writing to 192.168.201.22: rc = -22 [ 588.707219] LNet: Skipped 1 previous similar message [ 602.813058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 602.865195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 602.880518] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 602.895261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.22@tcp --health 500 [ 602.934142] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 609.142446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 609.166856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 609.176213] LNet: 67559:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 609.179339] LNetError: 67559:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 609.182316] LNetError: 67559:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 609.185035] LNet: Removed LNI 192.168.201.22@tcp [ 609.191330] LNet: Removed LNI 192.168.201.22@tcp1 [ 609.199396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 609.223637] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 609.234845] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 609.484187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 609.492960] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 609.782757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 609.800412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 609.824557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 609.838674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 609.853600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 609.869607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 609.885594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 609.900408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.22@tcp --health 0 [ 610.272374] LNet: There was an unexpected network error while writing to 192.168.201.22: rc = -22 [ 610.274485] LNet: Skipped 3 previous similar messages [ 611.013046] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 621.492916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 621.508846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 621.523687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 623.825988] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 06:54:20 (1781693660) [ 623.968366] LNet: 68614:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 623.971778] LNetError: 68614:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 623.975217] LNetError: 68614:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 623.979795] LNet: Removed LNI 192.168.201.22@tcp [ 623.992563] LNet: Removed LNI 192.168.201.22@tcp1 [ 624.221802] Key type .llcrypt unregistered [ 624.223482] Key type ._llcrypt unregistered [ 626.330728] Key type ._llcrypt registered [ 626.331976] Key type .llcrypt registered [ 626.398898] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 626.405912] alg: No test for adler32 (adler32-zlib) [ 627.215131] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 627.226581] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 630.769688] Lustre: DEBUG MARKER: Initial discovery [ 630.785912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.122@tcp [ 631.633914] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag [ 631.650834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 -e local_error [ 631.666562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.122@tcp [ 631.678867] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 631.692618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 631.709129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d 192.168.201.122@tcp -r 1 -m GET -l 3 [ 631.724369] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.201.122@tcp -r 1 -m GET -e local_error [ 631.740550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d 192.168.201.122@tcp -r 1 -m PUT -l 6 [ 632.250550] Lustre: DEBUG MARKER: Force oleg122-client.virtnet to discover 192.168.201.122@tcp (in background) [ 632.271689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.122@tcp [ 632.352287] LNet: 1 local NIs in recovery (showing 1): 192.168.201.22@tcp [ 632.776603] Lustre: DEBUG MARKER: Force oleg122-server to discover 192.168.201.22@tcp [ 633.656109] Lustre: DEBUG MARKER: Wait for 69367 [ 635.296236] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 635.772923] Lustre: DEBUG MARKER: Finished wait on 69367 [ 635.790627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay del -a [ 635.805539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 635.907894] LNet: 69823:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 635.910624] LNetError: 69823:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 635.919320] LNet: Removed LNI 192.168.201.22@tcp [ 636.063427] Key type .llcrypt unregistered [ 636.064733] Key type ._llcrypt unregistered [ 641.336749] Key type ._llcrypt registered [ 641.337570] Key type .llcrypt registered [ 641.819758] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 06:54:38 (1781693678) [ 641.998258] Key type .llcrypt unregistered [ 641.999765] Key type ._llcrypt unregistered [ 643.984615] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 644.045894] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 644.047820] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 644.050051] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 646.247350] Key type ._llcrypt registered [ 646.248838] Key type .llcrypt registered [ 646.317282] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 646.323310] alg: No test for adler32 (adler32-zlib) [ 647.135460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 647.159178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 647.169523] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 647.441148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 647.451945] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 650.330917] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 06:54:47 (1781693687) [ 650.384372] LNet: 71895:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 650.386722] LNetError: 71895:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 650.389861] LNet: Removed LNI 192.168.201.22@tcp [ 650.396606] LNet: Removed LNI 10.1.2.3@tcp [ 650.630344] Key type .llcrypt unregistered [ 650.631428] Key type ._llcrypt unregistered [ 652.766813] Key type ._llcrypt registered [ 652.768395] Key type .llcrypt registered [ 652.847304] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 652.853553] alg: No test for adler32 (adler32-zlib) [ 653.659591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 653.681163] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 653.693499] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 653.974119] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 653.984182] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 654.327294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 10.1.2.3@tcp 192.168.201.22@tcp [ 655.392123] LNetError: Unexpected error -22 connecting to 192.168.201.22@tcp at host 192.168.201.22:988 [ 656.622882] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 06:54:53 (1781693693) [ 656.689178] LNet: 73030:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 656.691653] LNetError: 73030:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 656.696341] LNet: Removed LNI 192.168.201.22@tcp [ 656.711577] LNet: Removed LNI 10.1.2.3@tcp [ 656.939524] Key type .llcrypt unregistered [ 656.941188] Key type ._llcrypt unregistered [ 659.105128] Key type ._llcrypt registered [ 659.106383] Key type .llcrypt registered [ 659.174902] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 659.181771] alg: No test for adler32 (adler32-zlib) [ 659.988335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 660.012106] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 660.021945] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 660.289446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 660.297257] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 660.620877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 192.168.201.22@tcp1 --nid 192.168.201.22@tcp2 [ 662.082511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.22@tcp1 192.168.201.22@tcp2 [ 662.093556] LNetError: 73864:0:(lib-move.c:2007:lnet_handle_find_routed_path()) Unable to send message from 192.168.201.22@tcp1 to 192.168.201.22@tcp2 - Route table may be misconfigured [ 662.103963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.22@tcp2 192.168.201.22@tcp1 [ 664.355451] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 06:55:01 (1781693701) [ 664.386402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 664.396828] LNet: 74432:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 664.400126] LNetError: 74432:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 664.404040] LNet: Removed LNI 192.168.201.22@tcp1 [ 665.446473] LNet: Removed LNI 192.168.201.22@tcp2 [ 665.455453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 665.477392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 665.487798] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 665.736423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 665.745810] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 666.057947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 666.078734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.22@tcp 192.168.201.22@tcp [ 666.097387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.22@tcp1 192.168.201.22@tcp1 [ 666.113191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.201.22@tcp -d 192.168.201.22@tcp -e network_timeout [ 666.127412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -e network_timeout [ 666.141461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -e network_timeout [ 666.156331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -e network_timeout [ 666.169170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 666.175773] LNet: There was an unexpected network error while writing to 192.168.201.22: rc = -22 [ 666.181798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 666.214739] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 666.249122] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 666.528096] LNet: 1 local NIs in recovery (showing 1): 192.168.201.22@tcp [ 668.350474] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 06:55:05 (1781693705) [ 668.384945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 668.396116] LNet: 75293:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 668.399132] LNetError: 75293:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 668.402268] LNetError: 75293:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 668.405136] LNet: Removed LNI 192.168.201.22@tcp [ 669.418486] LNet: Removed LNI 192.168.201.22@tcp1 [ 669.427290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 669.460565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 0@lo [ 669.562614] LNet: 75336:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 669.718129] Key type .llcrypt unregistered [ 669.719055] Key type ._llcrypt unregistered [ 673.559500] Key type ._llcrypt registered [ 673.560872] Key type .llcrypt registered [ 674.027176] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 06:55:10 (1781693710) [ 674.151935] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 674.157721] alg: No test for adler32 (adler32-zlib) [ 674.960772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 675.425824] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces [ 675.973330] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 06:55:12 (1781693712) [ 676.009016] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 676.018047] LNet: 76430:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 676.027720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 676.050963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 676.061115] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 676.315886] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 676.323567] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 676.640675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 676.656653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp1 [ 676.672339] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp1 [ 678.939191] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 06:55:15 (1781693715) [ 679.626809] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_220 Need at least 2 remote nodes found oleg122-server [ 680.244933] Lustre: DEBUG MARKER: == sanity-lnet test 221: Add routes w/DD disabled - check aliveness ========================================================== 06:55:17 (1781693717) [ 680.910069] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_221 Need at least 2 remote nodes found oleg122-server [ 681.504938] Lustre: DEBUG MARKER: == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 06:55:18 (1781693718) [ 682.133212] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_222 Need at least 2 remote nodes found oleg122-server [ 682.762133] Lustre: DEBUG MARKER: == sanity-lnet test 223: Check avoid_asym_router_failure=1 w/DD disabled ========================================================== 06:55:19 (1781693719) [ 683.397558] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_223 Need at least 2 remote nodes found oleg122-server [ 683.978863] Lustre: DEBUG MARKER: == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 06:55:20 (1781693720) [ 684.612769] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_224 Need at least 2 remote nodes found oleg122-server [ 685.121360] Lustre: DEBUG MARKER: == sanity-lnet test 225: Check avoid_asym_router_failure=0 w/DD disabled ========================================================== 06:55:22 (1781693722) [ 685.690181] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_225 Need at least 2 remote nodes found oleg122-server [ 686.258807] Lustre: DEBUG MARKER: == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 06:55:23 (1781693723) [ 686.875783] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_226 Need at least 3 remote nodes found oleg122-server [ 687.507696] Lustre: DEBUG MARKER: == sanity-lnet test 227: Check router peer health w/DD disabled ========================================================== 06:55:24 (1781693724) [ 688.208334] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_227 Need at least 3 remote nodes found oleg122-server [ 688.777572] Lustre: DEBUG MARKER: == sanity-lnet test 228: Routes should stay up when health is decremented ========================================================== 06:55:25 (1781693725) [ 689.411519] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_228 Need at least 3 remote nodes found oleg122-server [ 690.025800] Lustre: DEBUG MARKER: == sanity-lnet test 230: Test setting conns-per-peer ===== 06:55:26 (1781693726) [ 690.060640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 690.070503] LNet: 80300:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 690.073490] LNetError: 80300:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 690.075595] LNet: Removed LNI 192.168.201.22@tcp [ 691.110445] LNet: Removed LNI 192.168.201.22@tcp1 [ 691.120475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 691.144327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 691.157132] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 691.416740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 4 [ 691.448649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 691.515050] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 691.524452] LNet: 80453:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 691.527581] LNetError: 80453:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 691.530327] LNetError: 80453:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 691.538321] LNet: Removed LNI 192.168.201.22@tcp [ 691.546179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 691.568079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 5 [ 691.580352] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 691.857269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 691.917827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 692.966405] LNet: Removed LNI 192.168.201.22@tcp [ 692.975972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 692.998986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 693.010258] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 693.275932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 6 [ 693.308064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 693.377586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 693.386909] LNetError: 80745:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 693.388846] LNetError: 80745:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 693.395522] LNet: Removed LNI 192.168.201.22@tcp [ 693.403711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 693.428669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 7 [ 693.440275] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 693.733106] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 693.799172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 693.807497] LNet: 80891:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 693.810188] LNet: 80891:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 694.822417] LNet: Removed LNI 192.168.201.22@tcp [ 694.831162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 694.853527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 694.863356] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 695.160395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 8 [ 695.190762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 695.263974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 696.294361] LNet: Removed LNI 192.168.201.22@tcp [ 696.303226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 696.328696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 9 [ 696.339758] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 696.625498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 696.700283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 696.710760] LNetError: 81183:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 696.713866] LNetError: 81183:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 696.721365] LNet: Removed LNI 192.168.201.22@tcp [ 696.729831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 696.750780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 696.760248] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 697.027298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 10 [ 697.057618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 697.127775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 698.150365] LNet: Removed LNI 192.168.201.22@tcp [ 698.158189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 698.180199] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 11 [ 698.190277] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 698.489526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 698.569491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 698.577410] LNet: 81475:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 698.579942] LNet: 81475:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 699.622306] LNet: Removed LNI 192.168.201.22@tcp [ 699.631356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 699.653360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 699.666171] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 699.938684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 12 [ 699.970313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 700.047740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 701.094339] LNet: Removed LNI 192.168.201.22@tcp [ 701.102994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 701.125129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 13 [ 701.135372] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 701.412405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 702.537781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 702.546672] LNetError: 81777:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 702.549660] LNetError: 81777:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 702.557291] LNet: Removed LNI 192.168.201.22@tcp [ 702.564639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 702.588460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 702.597276] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 702.863226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 14 [ 702.895345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 702.988275] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 704.038376] LNet: Removed LNI 192.168.201.22@tcp [ 704.046511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 704.069304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 15 [ 704.078693] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 704.362079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 704.471874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 705.510572] LNet: Removed LNI 192.168.201.22@tcp [ 705.519418] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 705.543689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 705.556055] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 705.831350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 128 [ 705.859803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 705.878551] LNet: Removed LNI 192.168.201.22@tcp [ 705.887098] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 705.909473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 705.919222] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 706.189519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer -1 [ 708.571341] Lustre: DEBUG MARKER: == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 06:55:45 (1781693745) [ 708.597268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 708.606952] LNet: 82899:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 708.611329] LNet: 82899:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 708.620398] LNet: Removed LNI 192.168.201.22@tcp [ 708.631830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 708.655807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 [ 708.666716] LNet: Added LNI 192.168.201.22@tcp231 [8/256/0/180] [ 709.091271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 709.113299] LNet: Removed LNI 192.168.201.22@tcp231 [ 709.120855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 709.149803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 709.161307] LNet: Added LNI 192.168.201.22@tcp231 [8/256/0/180] [ 709.163989] LNet: 83051:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 709.309550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 709.326390] LNet: Removed LNI 192.168.201.22@tcp231 [ 709.332901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=-1 [ 709.343822] LNet: Added LNI 192.168.201.22@tcp231 [8/256/0/180] [ 709.751626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 709.770276] LNet: Removed LNI 192.168.201.22@tcp231 [ 709.777463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 709.803398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 709.813791] LNet: Added LNI 192.168.201.22@tcp231 [8/256/0/0] [ 709.815774] LNet: 83203:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 709.962457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 709.980655] LNet: Removed LNI 192.168.201.22@tcp231 [ 709.987851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=0 [ 709.999508] LNet: Added LNI 192.168.201.22@tcp231 [8/256/0/0] [ 710.408403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 710.422771] LNet: Removed LNI 192.168.201.22@tcp231 [ 710.432052] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 710.458238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 710.468933] LNet: Added LNI 192.168.201.22@tcp231 [8/256/0/60] [ 710.622017] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 710.638475] LNet: Removed LNI 192.168.201.22@tcp231 [ 710.645973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=60 [ 710.655927] LNet: Added LNI 192.168.201.22@tcp231 [8/256/0/60] [ 711.048615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 711.058344] LNetError: 83497:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 711.061055] LNetError: 83497:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 7 previous similar messages [ 711.067286] LNet: Removed LNI 192.168.201.22@tcp231 [ 711.079114] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 711.110519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 711.121197] LNet: Added LNI 192.168.201.22@tcp231 [8/256/0/180] [ 711.123726] LNet: 83505:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 711.126384] LNet: 83505:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 711.281022] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 711.297442] LNet: Removed LNI 192.168.201.22@tcp231 [ 711.304671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=180 [ 711.315238] LNet: Added LNI 192.168.201.22@tcp231 [8/256/0/180] [ 713.772452] Lustre: DEBUG MARKER: == sanity-lnet test 232: Test setting ToS value ========== 06:55:50 (1781693750) [ 713.802087] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 713.816635] LNet: Removed LNI 192.168.201.22@tcp231 [ 713.827241] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 713.859161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 [ 713.870546] LNet: Added LNI 192.168.201.22@tcp232 [8/256/0/180] [ 714.138629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 714.153914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 714.168561] LNet: Removed LNI 192.168.201.22@tcp232 [ 714.175685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos -1 [ 714.186595] LNet: Added LNI 192.168.201.22@tcp232 [8/256/0/180] [ 714.451758] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 714.465893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 714.477574] LNet: Removed LNI 192.168.201.22@tcp232 [ 714.484172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 104 [ 714.492277] LNet: Added LNI 192.168.201.22@tcp232 [8/256/0/180] [ 714.741143] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 714.754271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 714.766451] LNet: Removed LNI 192.168.201.22@tcp232 [ 714.772756] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 106 [ 714.782083] LNet: Added LNI 192.168.201.22@tcp232 [8/256/0/180] [ 715.045532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 715.060526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 715.078295] LNet: Removed LNI 192.168.201.22@tcp232 [ 715.086847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 300 [ 717.125431] Lustre: DEBUG MARKER: == sanity-lnet test 233: Check for successful resends ==== 06:55:54 (1781693754) [ 717.496993] Key type .llcrypt unregistered [ 717.498443] Key type ._llcrypt unregistered [ 719.542578] Key type ._llcrypt registered [ 719.543657] Key type .llcrypt registered [ 719.619150] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 719.626290] alg: No test for adler32 (adler32-zlib) [ 720.441475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 720.453859] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 723.453350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 726.341039] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 726.357377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 726.367242] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 726.871164] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_interrupt -r 2 [ 726.889119] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e local_interrupt -r 2 [ 726.903643] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_interrupt -r 2 [ 726.918834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e local_interrupt -r 2 [ 726.933596] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e local_interrupt -r 2 [ 726.950936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e local_interrupt -r 2 [ 726.968112] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e local_interrupt -r 2 [ 726.985085] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e local_interrupt -r 2 [ 727.001081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.22@tcp 192.168.201.122@tcp [ 727.012114] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 727.015588] LNet: 1 local NIs in recovery (showing 1): 192.168.201.22@tcp [ 727.101914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 727.116403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 727.130161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 727.144066] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 727.156770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 727.170421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 727.185238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 727.202292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 727.212310] LNet: 86000:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 727.219435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 727.233200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 727.248578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 727.261078] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 727.274869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 727.294326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 727.485296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_dropped -r 2 [ 727.506042] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e local_dropped -r 2 [ 727.524761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_dropped -r 2 [ 727.541340] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e local_dropped -r 2 [ 727.559268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e local_dropped -r 2 [ 727.578324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e local_dropped -r 2 [ 727.596554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e local_dropped -r 2 [ 727.613525] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e local_dropped -r 2 [ 727.630759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.22@tcp 192.168.201.122@tcp [ 727.640917] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 727.731304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 727.746776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 727.761541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 727.774985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 727.789948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 727.802588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 727.814496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 727.824555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 727.830738] LNet: 86121:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 727.838648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 727.855449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 727.871409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 727.882597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 727.894848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 727.907608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 728.098135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_aborted -r 2 [ 728.115572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e local_aborted -r 2 [ 728.130302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_aborted -r 2 [ 728.146535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e local_aborted -r 2 [ 728.161970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e local_aborted -r 2 [ 728.178225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e local_aborted -r 2 [ 728.194063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e local_aborted -r 2 [ 728.210864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e local_aborted -r 2 [ 728.226558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.22@tcp 192.168.201.122@tcp [ 728.316617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 728.328627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 728.341241] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 728.356772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 728.372602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 728.388428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 728.403173] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 728.419828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 728.433976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 728.446675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 728.458647] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 728.472314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 728.485550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 728.498588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 728.697136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_no_route -r 2 [ 728.713680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e local_no_route -r 2 [ 728.730542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_no_route -r 2 [ 728.747416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e local_no_route -r 2 [ 728.763786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e local_no_route -r 2 [ 728.781333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e local_no_route -r 2 [ 728.797939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e local_no_route -r 2 [ 728.816375] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e local_no_route -r 2 [ 728.832642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.22@tcp 192.168.201.122@tcp [ 728.842884] LNet: There was an unexpected network error while writing to 192.168.201.122: rc = -22 [ 728.845401] LNet: Skipped 2 previous similar messages [ 728.927542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 728.942723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 728.958440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 728.972062] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 728.984889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 728.999043] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 729.013044] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 729.027233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 729.035343] LNet: 86361:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 729.038404] LNet: 86361:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 729.047950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 729.062684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 729.076776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 729.090776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 729.105904] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 729.122560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 729.306956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e local_timeout -r 2 [ 729.323433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e local_timeout -r 2 [ 729.339163] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e local_timeout -r 2 [ 729.354281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e local_timeout -r 2 [ 729.370033] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e local_timeout -r 2 [ 729.388657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e local_timeout -r 2 [ 729.403842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e local_timeout -r 2 [ 729.420543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e local_timeout -r 2 [ 729.437080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.22@tcp 192.168.201.122@tcp [ 729.529361] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 729.544150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 729.556911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 729.569717] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 729.584068] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 729.598405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 729.611636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 729.624306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 729.638517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 729.652582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 729.667105] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 729.680233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 729.693292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 729.707177] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 729.895918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp -m GET -e remote_dropped -r 2 [ 729.911869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.122@tcp1 -m GET -e remote_dropped -r 2 [ 729.927707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp -m GET -e remote_dropped -r 2 [ 729.941624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp -d 192.168.201.22@tcp1 -m GET -e remote_dropped -r 2 [ 729.955792] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp -m GET -e remote_dropped -r 2 [ 729.971618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.122@tcp1 -m GET -e remote_dropped -r 2 [ 729.985954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp -m GET -e remote_dropped -r 2 [ 730.001698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.22@tcp1 -d 192.168.201.22@tcp1 -m GET -e remote_dropped -r 2 [ 730.017307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.22@tcp 192.168.201.122@tcp [ 730.027285] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.122@tcp [ 730.098218] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 730.111382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 730.124354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 730.138604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 730.150948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 730.163439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 730.173761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 730.187687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 730.203398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 730.217731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 730.232228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 730.243936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 730.257236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 730.272331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 731.441946] LNet: 86674:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 731.444435] LNetError: 86674:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 731.447615] LNet: Removed LNI 192.168.201.22@tcp [ 731.454277] LNet: Removed LNI 192.168.201.22@tcp1 [ 731.607579] Key type .llcrypt unregistered [ 731.609317] Key type ._llcrypt unregistered [ 737.016446] Key type ._llcrypt registered [ 737.018125] Key type .llcrypt registered [ 737.552258] Lustre: DEBUG MARKER: == sanity-lnet test 234: Verify NI status changes with manual health value transitions ========================================================== 06:56:14 (1781693774) [ 737.684353] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 737.694283] alg: No test for adler32 (adler32-zlib) [ 738.504232] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 738.526663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 738.537097] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 738.848170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 0 [ 738.861552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.22@tcp [ 738.870728] LNetError: 87650:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.201.22@tcp to recovery [ 738.904926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --nid 192.168.201.22@tcp [ 738.944386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 741.181957] Lustre: DEBUG MARKER: == sanity-lnet test 235: Verify NI status DOWN when health reaches 0 after failures ========================================================== 06:56:18 (1781693778) [ 741.213340] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 741.224146] LNet: 88232:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 741.227867] LNetError: 88232:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 741.236747] LNet: Removed LNI 192.168.201.22@tcp [ 741.255708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 741.290105] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 741.303882] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 741.565475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 741.575930] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 741.922427] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 100 --nid 192.168.201.22@tcp [ 741.932539] LNetError: 88505:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.201.22@tcp to recovery [ 742.304331] LNet: 1 local NIs in recovery (showing 1): 192.168.201.22@tcp [ 742.305044] LNet: There was an unexpected network error while writing to 192.168.201.22: rc = -22 [ 743.023944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 900 --nid 192.168.201.22@tcp [ 747.419946] Lustre: DEBUG MARKER: == sanity-lnet test 236: Local NI state propagates to routers ========================================================== 06:56:24 (1781693784) [ 748.109730] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_236 Need at least 2 remote nodes found oleg122-server [ 748.725333] Lustre: DEBUG MARKER: == sanity-lnet test 237: Check MR forwarding feature ===== 06:56:25 (1781693785) [ 749.360897] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_237 Need at least 2 remote nodes found oleg122-server [ 750.001867] Lustre: DEBUG MARKER: == sanity-lnet test 238: Check MR forwarding feature skipped for non-MR source ========================================================== 06:56:26 (1781693786) [ 750.656631] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_238 Need at least 2 remote nodes found oleg122-server [ 751.255338] Lustre: DEBUG MARKER: == sanity-lnet test 241: Check setting LND timeout value via lnetctl updates tunables ========================================================== 06:56:28 (1781693788) [ 751.290753] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 751.302544] LNet: 90129:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 751.306479] LNetError: 90129:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 751.310129] LNet: Removed LNI 192.168.201.22@tcp [ 751.315831] LNet: Removed LNI 192.168.201.22@tcp1 [ 751.326846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 751.349905] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 751.360626] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 751.663131] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set -h [ 751.692833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 51 [ 751.715791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 751.746816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 0 [ 751.765747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 751.785248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 50 [ 754.142538] Lustre: DEBUG MARKER: == sanity-lnet test 245: Check read-only params do not appear in export --backup ========================================================== 06:56:31 (1781693791) [ 754.189902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 754.200810] LNet: 90855:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 754.203421] LNetError: 90855:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 754.207134] LNetError: 90855:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 754.214718] LNet: Removed LNI 192.168.201.22@tcp [ 754.226582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 754.250362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 754.260368] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 756.663039] Lustre: DEBUG MARKER: == sanity-lnet test 250: test that linux routes are added ========================================================== 06:56:33 (1781693793) [ 756.695622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 756.705960] LNet: 91556:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 756.709459] LNetError: 91556:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 756.721499] LNet: Removed LNI 192.168.201.22@tcp [ 756.734119] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 756.759196] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 756.771231] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 759.245956] Lustre: DEBUG MARKER: == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 06:56:36 (1781693796) [ 759.774991] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_251 Need kfi NETTYPE [ 760.346103] Lustre: DEBUG MARKER: == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 06:56:37 (1781693797) [ 760.506565] LNet: Removed LNI 192.168.201.22@tcp [ 760.724573] Key type .llcrypt unregistered [ 760.725943] Key type ._llcrypt unregistered [ 762.671233] Key type ._llcrypt registered [ 762.672365] Key type .llcrypt registered [ 762.734321] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 762.740144] alg: No test for adler32 (adler32-zlib) [ 763.543272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 763.553403] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 766.341132] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 767.882197] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --timeout 15 192.168.201.122@tcp [ 767.990081] LNet: 93009:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 767.993144] LNetError: 93009:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 768.001358] LNet: Removed LNI 192.168.201.22@tcp [ 768.148346] Key type .llcrypt unregistered [ 768.149538] Key type ._llcrypt unregistered [ 771.566457] Key type ._llcrypt registered [ 771.567484] Key type .llcrypt registered [ 771.967807] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253 [ 772.441809] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254 [ 772.949720] Lustre: DEBUG MARKER: == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 06:56:49 (1781693809) [ 773.418815] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_255 kernel ip2nets deprecated in 2.17.53 [ 773.922327] Lustre: DEBUG MARKER: == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 06:56:50 (1781693810) [ 774.481384] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_256 Need at least 2 remote nodes found oleg122-server [ 774.978994] Lustre: DEBUG MARKER: == sanity-lnet test 257: Test graceful router shutdown/startup ========================================================== 06:56:51 (1781693811) [ 775.520559] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_257 Need at least 3 remote nodes found oleg122-server [ 776.029784] Lustre: DEBUG MARKER: == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 06:56:52 (1781693812) [ 776.500676] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 776.521955] alg: No test for adler32 (adler32-zlib) [ 777.375978] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 777.461483] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 779.048142] Key type lgssc registered [ 779.401518] Lustre: Echo OBD driver; http://www.lustre.org/ [ 786.759677] Key type lgssc unregistered [ 786.872366] LNet: 96177:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 786.874333] LNetError: 96177:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 786.885306] LNet: Removed LNI 192.168.201.22@tcp [ 787.101437] Key type .llcrypt unregistered [ 787.102435] Key type ._llcrypt unregistered [ 791.541766] Key type ._llcrypt registered [ 791.543478] Key type .llcrypt registered [ 791.976347] Lustre: DEBUG MARKER: == sanity-lnet test 265: Import of invalid CPT should fail ========================================================== 06:57:08 (1781693828) [ 792.104928] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 792.109849] alg: No test for adler32 (adler32-zlib) [ 792.908724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 792.929577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 792.945624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 792.960226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 792.976409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 794.945984] Lustre: DEBUG MARKER: == sanity-lnet test 266: Validate CPT parsing in network sequence ========================================================== 06:57:11 (1781693831) [ 794.981931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 794.990742] LNet: 97585:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 794.999954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 795.021464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 795.029859] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 795.316581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 795.323723] LNetError: 97732:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 795.333267] LNet: Removed LNI 192.168.201.22@tcp [ 795.340600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 795.358505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 795.366788] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 795.514303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 795.521168] LNet: 97760:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 795.523230] LNet: 97760:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 795.532360] LNet: Removed LNI 192.168.201.22@tcp [ 795.540467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 795.561302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 795.571291] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 797.572256] Lustre: DEBUG MARKER: == sanity-lnet test 270: Test NID metadata =============== 06:57:14 (1781693834) [ 797.600145] LNet: 98342:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 797.602781] LNetError: 98342:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 797.604845] LNetError: 98342:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 797.611443] LNet: Removed LNI 192.168.201.22@tcp [ 797.816564] Key type .llcrypt unregistered [ 797.818272] Key type ._llcrypt unregistered [ 799.688654] Key type ._llcrypt registered [ 799.689779] Key type .llcrypt registered [ 799.750124] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 799.755184] alg: No test for adler32 (adler32-zlib) [ 800.555669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 800.584331] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 800.586247] LNet: *** cfs_fail_loc=e003, val=100*** [ 800.815560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 800.823715] Ping reply received with mapping: [ 800.825161] found metadata reply NI[192.168.201.22@tcp] [ 800.826384] 16 c9 a8 c0 00 00 02 00 5a 5a 5a 5a 5a 5a 5a 5a [ 800.827619] 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a [ 800.828905] 5a 5a 5a 5a 5a 5a 5a 5a [ 800.872374] LNet: 98746:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 800.874387] LNetError: 98746:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 800.883339] LNet: Removed LNI 192.168.201.22@tcp [ 801.096495] Key type .llcrypt unregistered [ 801.098096] Key type ._llcrypt unregistered [ 804.434794] Key type ._llcrypt registered [ 804.435786] Key type .llcrypt registered [ 804.858726] Lustre: DEBUG MARKER: == sanity-lnet test 280: Don't panic when request_module fails ========================================================== 06:57:21 (1781693841) [ 805.007268] Key type .llcrypt unregistered [ 805.008239] Key type ._llcrypt unregistered [ 806.832445] Key type ._llcrypt registered [ 806.833927] Key type .llcrypt registered [ 806.890633] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 806.895957] alg: No test for adler32 (adler32-zlib) [ 807.754241] LNetError: 99790:0:(api-ni.c:2729:lnet_load_lnd()) Can't load LND tcp, module ksocklnd, rc=256 [ 807.873547] Key type .llcrypt unregistered [ 807.874767] Key type ._llcrypt unregistered [ 809.420787] Key type ._llcrypt registered [ 809.422336] Key type .llcrypt registered [ 809.844860] Lustre: DEBUG MARKER: == sanity-lnet test 290: Check that lnet acceptor is using explicit address binding ========================================================== 06:57:26 (1781693846) [ 810.001297] Key type .llcrypt unregistered [ 810.003116] Key type ._llcrypt unregistered [ 811.835345] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 811.837025] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 813.996817] Key type ._llcrypt registered [ 813.998043] Key type .llcrypt registered [ 814.052296] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 814.058099] alg: No test for adler32 (adler32-zlib) [ 814.855086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 814.873739] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 818.551812] Lustre: DEBUG MARKER: == sanity-lnet test 291: load lnet using custom accept port for bulk tcp connections ========================================================== 06:57:35 (1781693855) [ 818.582259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 818.591459] LNet: 101563:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 818.594468] LNetError: 101563:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 818.601575] LNet: Removed LNI 10.1.2.3@tcp [ 818.612330] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 818.849571] Key type .llcrypt unregistered [ 818.851065] Key type ._llcrypt unregistered [ 820.748316] Key type ._llcrypt registered [ 820.749325] Key type .llcrypt registered [ 820.803301] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 820.808536] alg: No test for adler32 (adler32-zlib) [ 821.615123] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 821.639374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 821.691215] LNet: 101856:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 821.694617] LNetError: 101856:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 821.704345] LNet: Removed LNI 192.168.201.22@tcp [ 821.925341] Key type .llcrypt unregistered [ 821.926884] Key type ._llcrypt unregistered [ 825.310281] Key type ._llcrypt registered [ 825.311339] Key type .llcrypt registered [ 825.733829] Lustre: DEBUG MARKER: == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 06:57:42 (1781693862) [ 825.935420] Key type .llcrypt unregistered [ 825.936705] Key type ._llcrypt unregistered [ 827.758981] Key type ._llcrypt registered [ 827.760165] Key type .llcrypt registered [ 827.823628] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 827.830059] alg: No test for adler32 (adler32-zlib) [ 830.628651] Lustre: DEBUG MARKER: == sanity-lnet test 301: Check for dynamic adds of same/wrong interface (memory leak) ========================================================== 06:57:47 (1781693867) [ 830.656742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 830.670782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 830.695985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 830.705926] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 830.930431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 830.943053] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if ens2 [ 830.961298] LNet: Removed LNI 192.168.201.22@tcp [ 831.039225] LNet: 103596:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 831.171275] Key type .llcrypt unregistered [ 831.172742] Key type ._llcrypt unregistered [ 834.547853] Key type ._llcrypt registered [ 834.549509] Key type .llcrypt registered [ 834.986948] Lustre: DEBUG MARKER: == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 06:57:51 (1781693871) [ 835.097496] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 835.101852] alg: No test for adler32 (adler32-zlib) [ 835.899246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 835.919416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 835.927884] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 836.163175] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 836.176652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug peer --nid 192.168.201.22@tcp [ 836.184354] LNet: Conn 0 [type, closing, crefcnt, srefcnt]: 2, 0, 1, 1 [ 836.186086] LNet: Conn 0 rx [scheduled, ready, state]: 0, 0, 1 [ 836.187456] LNet: Conn 0 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 836, 0, 0 [ 836.189614] LNet: Conn 0 sched [nconns, cpt]: 4, 0 [ 836.190763] LNet: Conn 1 [type, closing, crefcnt, srefcnt]: 3, 0, 1, 1 [ 836.192228] LNet: Conn 1 rx [scheduled, ready, state]: 0, 0, 1 [ 836.193656] LNet: Conn 1 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 836, 0, 0 [ 836.195624] LNet: Conn 1 sched [nconns, cpt]: 4, 0 [ 836.196647] LNet: Conn 2 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 836.198038] LNet: Conn 2 rx [scheduled, ready, state]: 0, 0, 1 [ 836.199283] LNet: Conn 2 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 836, 0, 886 [ 836.201017] LNet: Conn 2 sched [nconns, cpt]: 4, 0 [ 836.202092] LNet: Conn 3 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 836.203459] LNet: Conn 3 rx [scheduled, ready, state]: 0, 0, 1 [ 836.204740] LNet: Conn 3 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 836, 0, 886 [ 836.206561] LNet: Conn 3 sched [nconns, cpt]: 4, 0 [ 838.126639] Lustre: DEBUG MARKER: == sanity-lnet test 303: Check peer NI health after link down ========================================================== 06:57:55 (1781693875) [ 838.257783] LNet: 105134:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 838.260779] LNetError: 105134:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 838.269365] LNet: Removed LNI 192.168.201.22@tcp [ 838.466303] Key type .llcrypt unregistered [ 838.467570] Key type ._llcrypt unregistered [ 840.223037] Key type ._llcrypt registered [ 840.224543] Key type .llcrypt registered [ 840.286229] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 840.291816] alg: No test for adler32 (adler32-zlib) [ 841.089978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 841.099133] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 843.707638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 846.213143] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 846.225854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 846.234957] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 846.521251] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 846.581251] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 846.582496] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 846.583927] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 848.692453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp99 --if test1pg [ 848.700119] LNet: Added LNI 10.1.2.3@tcp99 [8/256/0/180] [ 851.757281] Lustre: DEBUG MARKER: == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 06:58:08 (1781693888) [ 851.787614] LNet: 106630:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 851.791086] LNetError: 106630:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 851.793742] LNet: Removed LNI 192.168.201.22@tcp [ 852.832088] LNet: Removed LNI 192.168.201.22@tcp1 [ 852.833745] LNetError: 106630:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 852.836970] LNetError: 106630:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 852.845589] LNet: Removed LNI 10.1.2.3@tcp99 [ 853.076357] Key type .llcrypt unregistered [ 853.077521] Key type ._llcrypt unregistered [ 855.174561] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 855.177361] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 857.337762] Key type ._llcrypt registered [ 857.338778] Key type .llcrypt registered [ 857.400288] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 857.408470] alg: No test for adler32 (adler32-zlib) [ 858.210523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 858.231721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 858.240436] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 858.489649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 858.497871] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 859.239704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.1.2.3@tcp --lock_prim [ 859.258056] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 859.279038] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 10.1.2.3@tcp --state 1048577 [ 859.291886] LNetError: 107379:0:(peer.c:2175:lnet_del_peer_ni()) peer 10.1.2.3@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 859.307503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 859.327084] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 859.471632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 859.499748] LNet: 107399:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 859.502273] LNetError: 107399:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 859.505075] LNet: Removed LNI 192.168.201.22@tcp [ 859.511322] LNet: Removed LNI 10.1.2.3@tcp [ 859.743339] Key type .llcrypt unregistered [ 859.744681] Key type ._llcrypt unregistered [ 863.470497] Key type ._llcrypt registered [ 863.471666] Key type .llcrypt registered [ 863.901734] Lustre: DEBUG MARKER: == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 06:58:20 (1781693900) [ 864.010211] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 864.015136] alg: No test for adler32 (adler32-zlib) [ 864.810801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 864.830249] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 864.837826] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 865.068362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.22@tcp [ 865.095754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping oleg122-client@tcp [ 866.930843] Lustre: DEBUG MARKER: == sanity-lnet test 310: Set timeout and verify ========== 06:58:23 (1781693903) [ 866.956155] LNet: 108921:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 866.959103] LNetError: 108921:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 866.969512] LNet: Removed LNI 192.168.201.22@tcp [ 867.177625] Key type .llcrypt unregistered [ 867.178653] Key type ._llcrypt unregistered [ 868.940732] Key type ._llcrypt registered [ 868.942334] Key type .llcrypt registered [ 868.996866] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 869.002564] alg: No test for adler32 (adler32-zlib) [ 869.794318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure --all [ 869.802136] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 869.814149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 871.721937] Lustre: DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 06:58:28 (1781693908) [ 872.170603] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type [ 872.680180] Lustre: DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 06:58:29 (1781693909) [ 873.150827] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type [ 873.669913] Lustre: DEBUG MARKER: == sanity-lnet test 350: Check import --exec of ping/discover ========================================================== 06:58:30 (1781693910) [ 873.697548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 873.706530] LNet: 110339:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 873.708994] LNetError: 110339:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 873.716578] LNet: Removed LNI 192.168.201.22@tcp [ 873.724627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 873.749204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 873.757645] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 874.006161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 874.022533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 876.084909] Lustre: DEBUG MARKER: == sanity-lnet test 400: Check for udsp add/delete net rule without net num ========================================================== 06:58:33 (1781693913) [ 876.113952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 876.124314] LNet: 111048:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 876.126672] LNetError: 111048:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 876.136650] LNet: Removed LNI 192.168.201.22@tcp [ 876.146923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 876.166428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --priority 0 [ 876.179899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 876.411581] Key type .llcrypt unregistered [ 876.412904] Key type ._llcrypt unregistered [ 879.919295] Key type ._llcrypt registered [ 879.920229] Key type .llcrypt registered [ 880.345821] Lustre: DEBUG MARKER: == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 06:58:37 (1781693917) [ 880.452489] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 880.457624] alg: No test for adler32 (adler32-zlib) [ 881.260929] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 881.280347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 881.288842] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 881.511027] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 881.529903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 883.575792] Lustre: DEBUG MARKER: == sanity-lnet test 402: Destination net rule should not panic ========================================================== 06:58:40 (1781693920) [ 883.613131] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 883.621963] LNet: 112605:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 883.624721] LNetError: 112605:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 883.632886] LNet: Removed LNI 192.168.201.22@tcp [ 883.642683] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 883.662714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst kfi --priority 0 [ 883.673450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 402@kfi [ 885.600146] Lustre: DEBUG MARKER: == sanity-lnet test 403: Add and delete udsp net and nid rules ========================================================== 06:58:42 (1781693922) [ 885.629961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 885.638556] LNet: 113181:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 885.648772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 885.675384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 885.790729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 885.801192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 885.821513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src 1.1.1.1@tcp --prio 1 [ 885.940274] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 885.951192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 885.973710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 885.988456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 886.014940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 886.133039] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 886.146828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 886.168938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst 1.1.1.1@tcp --prio 1 [ 886.319385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 886.329819] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 888.310903] Lustre: DEBUG MARKER: == sanity-lnet test 404: Check udsp src/dst net rule ===== 06:58:45 (1781693925) [ 888.345318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 888.354314] LNet: 113817:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 888.357721] LNet: 113817:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 888.369103] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 888.389141] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 888.398711] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 888.631669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 888.639085] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 888.920758] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 888.941728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 889.014161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 889.022403] LNetError: 114099:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 889.024678] LNet: Removed LNI 192.168.201.22@tcp [ 889.031306] LNet: Removed LNI 192.168.201.22@tcp2 [ 889.038605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 889.061446] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 889.063826] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 889.066136] LNet: 114105:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 889.227751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 889.291739] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 890.336127] LNet: Removed LNI 192.168.201.22@tcp [ 890.337496] LNetError: 114142:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 890.339715] LNetError: 114142:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 890.348268] LNet: Removed LNI 192.168.201.22@tcp2 [ 890.357130] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 890.381211] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 890.391266] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 890.631926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 890.639937] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 890.943110] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 890.959781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 890.981497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 891.052412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 891.059988] LNet: 114426:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 891.062684] LNet: 114426:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 892.128112] LNet: Removed LNI 192.168.201.22@tcp [ 893.158335] LNet: Removed LNI 192.168.201.22@tcp2 [ 893.167580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 893.190506] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 893.193234] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 893.195716] LNet: 114433:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 893.345475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 893.410449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 893.418366] LNetError: 114470:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 893.421267] LNetError: 114470:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 893.424523] LNet: Removed LNI 192.168.201.22@tcp [ 894.438289] LNet: Removed LNI 192.168.201.22@tcp2 [ 894.445731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 894.467180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 894.477777] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 894.702867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 894.710811] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 894.988438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 895.007136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 895.073143] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 895.082321] LNet: 114749:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 895.085283] LNet: 114749:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 896.096125] LNet: Removed LNI 192.168.201.22@tcp [ 896.103735] LNet: Removed LNI 192.168.201.22@tcp2 [ 896.111322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 896.132344] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 896.135226] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 896.136869] LNet: 114755:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 896.285697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 896.348510] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 897.376110] LNet: Removed LNI 192.168.201.22@tcp [ 897.383288] LNet: Removed LNI 192.168.201.22@tcp2 [ 897.390495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 897.409730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 897.417551] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 897.652019] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 897.659928] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 897.948994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 897.962775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 897.980680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 898.047252] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 898.055752] LNetError: 115077:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 898.058575] LNetError: 115077:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 898.061918] LNet: Removed LNI 192.168.201.22@tcp [ 899.110311] LNet: Removed LNI 192.168.201.22@tcp2 [ 899.117206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 899.136849] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 899.138933] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 899.141125] LNet: 115083:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 899.284810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 899.347382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 899.355217] LNet: Removed LNI 192.168.201.22@tcp [ 900.390257] LNet: Removed LNI 192.168.201.22@tcp2 [ 900.397484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 900.415495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 900.424544] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 900.651136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 900.658707] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 900.920333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 900.932200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 900.952333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 901.069447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 902.112150] LNet: Removed LNI 192.168.201.22@tcp [ 902.119259] LNet: Removed LNI 192.168.201.22@tcp2 [ 902.126311] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 902.147291] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 902.149774] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 902.298685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 902.407985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 903.456124] LNet: Removed LNI 192.168.201.22@tcp [ 903.463306] LNet: Removed LNI 192.168.201.22@tcp2 [ 903.470284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 903.489564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 903.497966] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 903.736226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 903.743745] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 904.042975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 904.059621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 904.073539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 904.100741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 904.228452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 904.237644] LNet: 115776:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 904.240612] LNet: 115776:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 905.248135] LNet: Removed LNI 192.168.201.22@tcp [ 906.278354] LNet: Removed LNI 192.168.201.22@tcp2 [ 906.289288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 906.311941] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 906.314352] LNet: Added LNI 192.168.201.22@tcp2 [8/256/0/180] [ 906.316874] LNet: 115782:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 906.319473] LNet: 115782:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 906.489411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 908.805727] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_410 skipping ALWAYS excluded test 410 [ 909.342428] Lustre: DEBUG MARKER: == sanity-lnet test 450: Check import of multiple interfaces ========================================================== 06:59:06 (1781693946) [ 909.367375] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 909.376334] LNetError: 116536:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 909.378951] LNetError: 116536:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 9 previous similar messages [ 909.381701] LNet: Removed LNI 192.168.201.22@tcp [ 910.438518] LNet: Removed LNI 192.168.201.22@tcp2 [ 910.446748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 910.545261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 910.555291] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 910.801696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 910.810391] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 911.098410] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 911.107410] LNet: Removed LNI 192.168.201.22@tcp [ 911.119316] LNet: Removed LNI 10.1.2.3@tcp [ 911.126808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 911.149254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-450.yaml [ 911.158391] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 911.160153] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 911.301578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del /tmp/sanity-lnet-450.yaml [ 911.311159] LNet: Removed LNI 192.168.201.22@tcp [ 911.323366] LNet: Removed LNI 10.1.2.3@tcp [ 913.390473] Lustre: DEBUG MARKER: == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 06:59:10 (1781693950) [ 913.427628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 913.443810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 913.478055] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 913.521127] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 913.522867] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 913.524925] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 915.656980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 915.665646] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 915.898282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 915.908888] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 916.585737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.22@tcp [ 916.600920] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d *@tcp -r 1 -l 1 -m PUT [ 916.622267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 192.168.201.22@tcp -m PUT -e local_timeout -r 1 [ 916.644926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 10.1.2.3@tcp -m PUT -e local_timeout -r 1 [ 916.665709] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 916.667226] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 916.749276] LNetError: 117479:0:(lib-move.c:4714:lnet_parse()) 192.168.201.22@tcp, src 192.168.201.22@tcp: Dropping PUT (error -108 looking up sender) [ 918.816099] LNet: Removed LNI 192.168.201.22@tcp [ 918.823253] LNet: Removed LNI 10.1.2.3@tcp [ 918.966835] Key type .llcrypt unregistered [ 918.968133] Key type ._llcrypt unregistered [ 922.537198] Key type ._llcrypt registered [ 922.538371] Key type .llcrypt registered [ 922.982938] Lustre: DEBUG MARKER: == sanity-lnet test 501: Verify lnetctl peer set --health (SR) ========================================================== 06:59:19 (1781693959) [ 923.102786] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 923.108041] alg: No test for adler32 (adler32-zlib) [ 923.907533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 924.026907] LNet: 118790:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 924.230382] Key type .llcrypt unregistered [ 924.231467] Key type ._llcrypt unregistered [ 926.167948] Key type ._llcrypt registered [ 926.169358] Key type .llcrypt registered [ 926.237695] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 926.244358] alg: No test for adler32 (adler32-zlib) [ 927.048598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 927.059221] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 929.910190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 929.973708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 930.177954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.122@tcp [ 930.265678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 930.278530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 930.291493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 930.304297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 930.318318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 930.332270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 930.347445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 930.361681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 930.369901] LNet: 119238:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 930.377319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 930.388809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 930.398760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 930.410304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 930.424331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 930.438812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 930.664080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 930.745025] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 930.758128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 930.776032] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 930.790678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 930.803614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 930.817634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 930.833224] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 930.848544] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 930.861946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 930.875188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 930.889308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 930.902853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 930.915209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 930.927468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 931.146048] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.122@tcp [ 931.169230] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.122@tcp [ 931.225390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 931.237828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 931.252626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 931.265845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 931.280396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 931.294291] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 931.307489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 931.320881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 931.328712] LNet: 119472:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 931.332071] LNet: 119472:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 931.340935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 931.353337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 931.364928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 931.377251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 931.390093] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 931.401152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 931.620087] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 931.699324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 931.712360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 931.723865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 931.737463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 931.751373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 931.764527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 931.779819] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 931.794878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 931.808424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 931.821433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 931.833986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 931.846067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 931.858543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 931.872289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 931.978331] LNet: 119635:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 931.980770] LNetError: 119635:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 931.987334] LNet: Removed LNI 192.168.201.22@tcp [ 932.129415] Key type .llcrypt unregistered [ 932.130412] Key type ._llcrypt unregistered [ 937.270747] Key type ._llcrypt registered [ 937.271719] Key type .llcrypt registered [ 937.709782] Lustre: DEBUG MARKER: == sanity-lnet test 502: Verify lnetctl peer set --health (MR) ========================================================== 06:59:34 (1781693974) [ 937.827481] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 937.833426] alg: No test for adler32 (adler32-zlib) [ 938.635168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 938.772558] LNet: 120521:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 939.001355] Key type .llcrypt unregistered [ 939.002721] Key type ._llcrypt unregistered [ 940.998550] Key type ._llcrypt registered [ 940.999900] Key type .llcrypt registered [ 941.066514] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 941.071837] alg: No test for adler32 (adler32-zlib) [ 941.876728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 941.889263] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 944.665826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.122@tcp [ 947.333709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 947.348313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 947.356715] LNet: Added LNI 192.168.201.22@tcp1 [8/256/0/180] [ 947.654297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 947.851629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.122@tcp [ 947.934398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 947.945704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 947.958814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 947.972061] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 947.983186] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 947.995572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 948.008590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 948.022496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 948.030222] LNet: 121170:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 948.038574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 948.051483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 948.064196] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.122@tcp [ 948.064685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 948.078891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 948.092654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 948.106470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 948.313583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 948.393544] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 948.405539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 948.417737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 948.430715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 948.444954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 948.458221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 948.470772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 948.483942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 948.497143] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 948.510080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 948.520910] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 948.532591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 948.544658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 948.558778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 948.769337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.122@tcp [ 948.847789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 948.859038] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 948.871527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 948.882595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 948.893334] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 948.905410] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 948.919352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 948.932803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 948.942270] LNet: 121404:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 948.945856] LNet: 121404:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 948.954061] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 948.967355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 948.979938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 948.993221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 949.005524] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 949.018943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 949.241151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 949.317713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 949.328513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 949.340374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 949.352414] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 949.363202] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 949.375101] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 949.388218] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 949.402467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 949.415166] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 949.427279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 949.441417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 949.453212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 949.465493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 949.477298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 951.585841] LNet: 121602:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 951.589201] LNetError: 121602:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 951.593361] LNet: Removed LNI 192.168.201.22@tcp [ 951.601340] LNet: Removed LNI 192.168.201.22@tcp1 [ 951.728679] Key type .llcrypt unregistered [ 951.729856] Key type ._llcrypt unregistered [ 956.905165] Key type ._llcrypt registered [ 956.906470] Key type .llcrypt registered [ 957.359507] Lustre: DEBUG MARKER: == sanity-lnet test 600: lnet_legacy2yaml: basic ip2nets conversion ========================================================== 06:59:54 (1781693994) [ 959.661317] Lustre: DEBUG MARKER: == sanity-lnet test 601: lnet_legacy2yaml: basic routes conversion ========================================================== 06:59:56 (1781693996) [ 961.928243] Lustre: DEBUG MARKER: == sanity-lnet test 602: lnet_legacy2yaml: routes with priority ========================================================== 06:59:58 (1781693998) [ 964.147206] Lustre: DEBUG MARKER: == sanity-lnet test 603: lnet_legacy2yaml: default hop flag ========================================================== 07:00:01 (1781694001) [ 966.372097] Lustre: DEBUG MARKER: == sanity-lnet test 604: lnet_legacy2yaml: default hop with explicit value ========================================================== 07:00:03 (1781694003) [ 968.806095] Lustre: DEBUG MARKER: == sanity-lnet test 605: lnet_legacy2yaml: quote leading star in IP pattern ========================================================== 07:00:05 (1781694005) [ 971.064754] Lustre: DEBUG MARKER: == sanity-lnet test 606: lnet_legacy2yaml: file inputs === 07:00:07 (1781694007) [ 973.330966] Lustre: DEBUG MARKER: == sanity-lnet test 607: lnet_legacy2yaml: options line with combined ip2nets and routes ========================================================== 07:00:10 (1781694010) [ 975.460252] Lustre: DEBUG MARKER: == sanity-lnet test 608: lnet_legacy2yaml: error on invalid rule (missing IP pattern) ========================================================== 07:00:12 (1781694012) [ 977.561361] Lustre: DEBUG MARKER: == sanity-lnet test 609: lnet_legacy2yaml: error on no input ========================================================== 07:00:14 (1781694014) [ 979.600556] Lustre: DEBUG MARKER: == sanity-lnet test 610: lnet_legacy2yaml: multiple files and inline, check order ========================================================== 07:00:16 (1781694016) [ 981.721238] Lustre: DEBUG MARKER: == sanity-lnet test 611: lnet_legacy2yaml: invalid hopcount ========================================================== 07:00:18 (1781694018) [ 983.799568] Lustre: DEBUG MARKER: == sanity-lnet test 612: lnet_legacy2yaml: malformed IP pattern ========================================================== 07:00:20 (1781694020) [ 986.085866] Lustre: DEBUG MARKER: == sanity-lnet test 613: lnet_legacy2yaml: multiple IP patterns in single rule ========================================================== 07:00:22 (1781694022) [ 988.613158] Lustre: DEBUG MARKER: == sanity-lnet test 614: lnet_legacy2yaml: interfaces enumeration with multiple IP ranges ========================================================== 07:00:25 (1781694025) [ 990.841891] Lustre: DEBUG MARKER: == sanity-lnet test 615: lnet_legacy2yaml: modprobe configuration file input ========================================================== 07:00:27 (1781694027) [ 993.001559] Lustre: DEBUG MARKER: == sanity-lnet test 616: lnet_legacy2yaml: combining modprobe file with explicit ip2nets file ========================================================== 07:00:29 (1781694029) [ 995.292991] Lustre: DEBUG MARKER: == sanity-lnet test 617: lnet_legacy2yaml: round-trip tunables from LNet to modprobe to YAML ========================================================== 07:00:32 (1781694032) [ 995.408392] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 995.414873] alg: No test for adler32 (adler32-zlib) [ 996.212325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 996.252478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 996.260473] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 996.633477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 996.640057] LNet: 132299:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 996.641969] LNetError: 132299:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 996.651222] LNet: Removed LNI 192.168.201.22@tcp [ 996.657236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 996.671805] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 996.678646] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 998.677181] Lustre: DEBUG MARKER: == sanity-lnet test 618: lnet_legacy2yaml: alternative route format ========================================================== 07:00:35 (1781694035) [ 1000.833442] Lustre: DEBUG MARKER: == sanity-lnet test 619: lnet_legacy2yaml: multiple alternative format routes from file ========================================================== 07:00:37 (1781694037) [ 1003.167050] Lustre: DEBUG MARKER: == sanity-lnet test 620: lnet_legacy2yaml: mixed legacy and alternative route formats ========================================================== 07:00:40 (1781694040) [ 1005.395157] Lustre: DEBUG MARKER: == sanity-lnet test 621: lnet_legacy2yaml: alternative format with invalid hopcount ========================================================== 07:00:42 (1781694042) [ 1007.449599] Lustre: DEBUG MARKER: == sanity-lnet test 622: lnet_legacy2yaml: alternative format with default-hop flag ========================================================== 07:00:44 (1781694044) [ 1009.530657] Lustre: DEBUG MARKER: == sanity-lnet test 623: lnet_legacy2yaml: basic networks conversion ========================================================== 07:00:46 (1781694046) [ 1011.639148] Lustre: DEBUG MARKER: == sanity-lnet test 624: lnet_legacy2yaml: networks with interfaces ========================================================== 07:00:48 (1781694048) [ 1013.743699] Lustre: DEBUG MARKER: == sanity-lnet test 625: lnet_legacy2yaml: multiple networks ========================================================== 07:00:50 (1781694050) [ 1016.029710] Lustre: DEBUG MARKER: == sanity-lnet test 626: lnet_legacy2yaml: mixed networks, ip2nets, and routes ========================================================== 07:00:52 (1781694052) [ 1018.347166] Lustre: DEBUG MARKER: == sanity-lnet test 627: lnet_legacy2yaml: invalid networks format ========================================================== 07:00:55 (1781694055) [ 1020.594610] Lustre: DEBUG MARKER: == sanity-lnet test 628: lnet_legacy2yaml: missing delimiter error ========================================================== 07:00:57 (1781694057) [ 1022.807412] Lustre: DEBUG MARKER: == sanity-lnet test 629: lnet_legacy2yaml: multiple gateways error ========================================================== 07:00:59 (1781694059) [ 1024.994896] Lustre: DEBUG MARKER: == sanity-lnet test complete, duration 915 sec =========== 07:01:01 (1781694061) [ 1025.079644] LNet: 139765:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1025.082309] LNetError: 139765:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1025.091396] LNet: Removed LNI 192.168.201.22@tcp [ 1025.316432] Key type .llcrypt unregistered [ 1025.317763] Key type ._llcrypt unregistered [ 1028.912104] Key type ._llcrypt registered [ 1028.913505] Key type .llcrypt registered [ 1029.119289] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1029.124792] alg: No test for adler32 (adler32-zlib) [ 1029.980149] Lustre: Lustre: Build Version: 2.17.54_40_g42483b8 [ 1030.065535] LNet: Added LNI 192.168.201.22@tcp [8/256/0/180] [ 1031.648156] Key type lgssc registered [ 1032.058309] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1053.536577] Lustre: Mounted lustre-client [ 1056.015147] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1058.240060] Lustre: Unmounted lustre-client [ 1069.956548] Key type lgssc unregistered [ 1070.068556] LNet: 142678:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1070.071120] LNetError: 142678:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1070.080792] LNet: Removed LNI 192.168.201.22@tcp [ 1070.323109] Key type .llcrypt unregistered [ 1070.324332] Key type ._llcrypt unregistered