[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 438758776 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54d0-0x000f54df] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895288K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003241] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008711] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.010019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.012008] pid_max: default: 32768 minimum: 301 [ 0.013132] LSM: Security Framework initializing [ 0.014058] Yama: becoming mindful. [ 0.015031] SELinux: Initializing. [ 0.016082] *** VALIDATE selinux *** [ 0.024693] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.029255] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.030161] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032082] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.033114] *** VALIDATE tmpfs *** [ 0.035469] *** VALIDATE proc *** [ 0.036244] *** VALIDATE cgroup *** [ 0.037010] *** VALIDATE cgroup2 *** [ 0.038293] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.040044] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.041008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.042031] Spectre V2 : User space: Vulnerable [ 0.043008] Speculative Store Bypass: Vulnerable [ 0.046448] debug: unmapping init [mem 0xffffffffa8059000-0xffffffffa8060fff] [ 0.049179] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.050697] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.051027] ... version: 2 [ 0.052014] ... bit width: 48 [ 0.053013] ... generic registers: 4 [ 0.054015] ... value mask: 0000ffffffffffff [ 0.055016] ... max period: 00007fffffffffff [ 0.056026] ... fixed-purpose events: 3 [ 0.057015] ... event mask: 000000070000000f [ 0.058378] rcu: Hierarchical SRCU implementation. [ 0.060486] smp: Bringing up secondary CPUs ... [ 0.061632] x86: Booting SMP configuration: [ 0.062027] .... node #0, CPUs: #1 #2 #3 [ 0.065233] smp: Brought up 1 node, 4 CPUs [ 0.067014] smpboot: Max logical packages: 1 [ 0.068018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.134546] node 0 deferred pages initialised in 64ms [ 0.136033] devtmpfs: initialized [ 0.136986] x86/mm: Memory block size: 128MB [ 0.138196] gcov: version magic: 0x41383552 [ 0.139651] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.140095] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.141338] pinctrl core: initialized pinctrl subsystem [ 0.142153] [ 0.142528] ************************************************************* [ 0.143012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.144008] ** ** [ 0.145010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.146008] ** ** [ 0.147010] ** This means that this kernel is built to expose internal ** [ 0.148007] ** IOMMU data structures, which may compromise security on ** [ 0.149009] ** your system. ** [ 0.150008] ** ** [ 0.151008] ** If you see this message and you are not debugging the ** [ 0.152010] ** kernel, report this immediately to your vendor! ** [ 0.153009] ** ** [ 0.154008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.155009] ************************************************************* [ 0.156556] NET: Registered protocol family 16 [ 0.157330] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.158061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.159054] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.160445] cpuidle: using governor menu [ 0.164740] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.171713] PCI: Using configuration type 1 for base access [ 0.175192] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.192054] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.193036] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.195100] cryptd: max_cpu_qlen set to 1000 [ 0.198277] ACPI: Added _OSI(Module Device) [ 0.200023] ACPI: Added _OSI(Processor Device) [ 0.201012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.203014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.208251] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.213439] ACPI: Interpreter enabled [ 0.215064] ACPI: PM: (supports S0 S3 S4 S5) [ 0.216011] ACPI: Using IOAPIC for interrupt routing [ 0.218105] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.221390] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.233447] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.235048] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.238022] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.241078] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.246480] acpiphp: Slot [2] registered [ 0.248152] acpiphp: Slot [5] registered [ 0.249139] acpiphp: Slot [6] registered [ 0.251131] acpiphp: Slot [3] registered [ 0.252181] acpiphp: Slot [4] registered [ 0.254143] acpiphp: Slot [7] registered [ 0.255194] acpiphp: Slot [8] registered [ 0.257113] acpiphp: Slot [9] registered [ 0.259070] acpiphp: Slot [10] registered [ 0.260081] acpiphp: Slot [11] registered [ 0.261159] acpiphp: Slot [12] registered [ 0.263180] acpiphp: Slot [13] registered [ 0.265135] acpiphp: Slot [14] registered [ 0.268285] acpiphp: Slot [15] registered [ 0.269105] acpiphp: Slot [16] registered [ 0.271093] acpiphp: Slot [17] registered [ 0.272116] acpiphp: Slot [18] registered [ 0.275131] acpiphp: Slot [19] registered [ 0.277125] acpiphp: Slot [20] registered [ 0.278233] acpiphp: Slot [21] registered [ 0.280133] acpiphp: Slot [22] registered [ 0.282258] acpiphp: Slot [23] registered [ 0.285216] acpiphp: Slot [24] registered [ 0.287116] acpiphp: Slot [25] registered [ 0.289098] acpiphp: Slot [26] registered [ 0.291175] acpiphp: Slot [27] registered [ 0.293182] acpiphp: Slot [28] registered [ 0.295153] acpiphp: Slot [29] registered [ 0.296132] acpiphp: Slot [30] registered [ 0.298105] acpiphp: Slot [31] registered [ 0.299083] PCI host bridge to bus 0000:00 [ 0.300020] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.303086] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.305022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.308031] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.310025] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.312022] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.314178] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.316099] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.319219] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.326016] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.330025] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.332019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.335019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.337017] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.339482] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.341894] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.343039] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.345591] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.349012] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.358015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.362009] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.365902] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.371020] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.378015] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.401017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.417518] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.422017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.426013] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.437017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.447469] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.449447] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.451613] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.454515] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.457292] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.463115] iommu: Default domain type: Passthrough [ 0.465541] SCSI subsystem initialized [ 0.467128] ACPI: bus type USB registered [ 0.469148] usbcore: registered new interface driver usbfs [ 0.471086] usbcore: registered new interface driver hub [ 0.473109] usbcore: registered new device driver usb [ 0.475206] pps_core: LinuxPPS API ver. 1 registered [ 0.477012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.480080] PTP clock support registered [ 0.482305] EDAC MC: Ver: 3.0.0 [ 0.483617] PCI: Using ACPI for IRQ routing [ 0.484903] NetLabel: Initializing [ 0.485011] NetLabel: domain hash size = 128 [ 0.486010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.487148] NetLabel: unlabeled traffic allowed by default [ 0.489127] vgaarb: loaded [ 0.491363] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.493014] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.501802] clocksource: Switched to clocksource kvm-clock [ 0.607064] VFS: Disk quotas dquot_6.6.0 [ 0.608701] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.610972] *** VALIDATE ramfs *** [ 0.612229] *** VALIDATE hugetlbfs *** [ 0.613720] pnp: PnP ACPI init [ 0.615687] pnp: PnP ACPI: found 6 devices [ 0.633312] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.636766] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.639064] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.641119] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.643773] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.646783] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.649429] NET: Registered protocol family 2 [ 0.651863] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.656273] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.659656] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.664755] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.668222] TCP: Hash tables configured (established 65536 bind 65536) [ 0.671197] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.674363] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.677337] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.680425] NET: Registered protocol family 1 [ 0.684341] RPC: Registered named UNIX socket transport module. [ 0.687061] RPC: Registered udp transport module. [ 0.688919] RPC: Registered tcp transport module. [ 0.690652] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.693160] NET: Registered protocol family 44 [ 0.695018] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.697311] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.699714] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.702081] PCI: CLS 0 bytes, default 64 [ 0.703988] Unpacking initramfs... [ 2.200839] debug: unmapping init [mem 0xffffa004bcc64000-0xffffa004bffcffff] [ 2.205071] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.207461] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.210724] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.699454] Initialise system trusted keyrings [ 2.701301] Key type blacklist registered [ 2.703725] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.713285] zbud: loaded [ 2.716459] *** VALIDATE nfs *** [ 2.717715] *** VALIDATE nfs4 *** [ 2.719606] pstore: using deflate compression [ 2.723719] Platform Keyring initialized [ 2.832655] NET: Registered protocol family 38 [ 2.834456] Key type asymmetric registered [ 2.835641] Asymmetric key parser 'x509' registered [ 2.837388] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.840253] io scheduler mq-deadline registered [ 2.841758] io scheduler kyber registered [ 2.843268] io scheduler bfq registered [ 2.844780] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.846640] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.848423] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.850350] ACPI: Power Button [PWRF] [ 2.855060] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.860848] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.870704] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.897249] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.925462] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.929502] Non-volatile memory driver v1.3 [ 2.931153] Linux agpgart interface v0.103 [ 2.960152] virtio_blk virtio1: [vda] 150008 512-byte logical blocks (76.8 MB/73.2 MiB) [ 2.963193] vda: detected capacity change from 0 to 76804096 [ 2.979473] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.982496] vdb: detected capacity change from 0 to 1073741824 [ 2.989179] libphy: Fixed MDIO Bus: probed [ 3.005824] usbcore: registered new interface driver usbserial_generic [ 3.008661] usbserial: USB Serial support registered for generic [ 3.015727] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.025880] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.027950] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.032648] mousedev: PS/2 mouse device common for all mice [ 3.038800] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.039368] rtc_cmos 00:05: RTC can wake from S4 [ 3.057623] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.062741] rtc_cmos 00:05: registered as rtc0 [ 3.068327] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.076524] intel_pstate: CPU model not supported [ 3.089187] hid: raw HID events driver (C) Jiri Kosina [ 3.097512] usbcore: registered new interface driver usbhid [ 3.098271] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.107398] usbhid: USB HID core driver [ 3.108232] drop_monitor: Initializing network drop monitor service [ 3.128969] Initializing XFRM netlink socket [ 3.133286] NET: Registered protocol family 10 [ 3.144356] Segment Routing with IPv6 [ 3.145822] NET: Registered protocol family 17 [ 3.148347] mpls_gso: MPLS GSO support [ 3.154860] RAS: Correctable Errors collector initialized. [ 3.157411] AVX version of gcm_enc/dec engaged. [ 3.158967] AES CTR mode by8 optimization enabled [ 3.410900] sched_clock: Marking stable (3410873453, 0)->(4302944718, -892071265) [ 3.418206] registered taskstats version 1 [ 3.429828] Loading compiled-in X.509 certificates [ 3.439671] zswap: loaded using pool lzo/zbud [ 3.520821] Key type big_key registered [ 3.543690] Key type encrypted registered [ 3.545377] ima: No TPM chip found, activating TPM-bypass! [ 3.548892] ima: Allocated hash algorithm: sha1 [ 3.551254] ima: No architecture policies found [ 3.553933] evm: Initialising EVM extended attributes: [ 3.558987] evm: security.selinux [ 3.560873] evm: security.ima [ 3.562780] evm: security.capability [ 3.564917] evm: HMAC attrs: 0x1 [ 3.568760] rtc_cmos 00:05: setting system clock to 2026-09-15 04:09:50 UTC (1789445390) [ 3.582291] debug: unmapping init [mem 0xffffffffa9003000-0xffffffffa91fffff] [ 3.588708] debug: unmapping init [mem 0xffffffffa7d82000-0xffffffffa8058fff] [ 3.603247] Write protecting the kernel read-only data: 28672k [ 3.622712] debug: unmapping init [mem 0xffffffffa6403000-0xffffffffa65fffff] [ 3.633712] debug: unmapping init [mem 0xffffffffa6d14000-0xffffffffa6dfffff] [ 3.686720] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.705922] systemd[1]: Detected virtualization kvm. [ 3.711803] systemd[1]: Detected architecture x86-64. [ 3.714778] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.765988] systemd[1]: No hostname configured. [ 3.771798] systemd[1]: Set hostname to . [ 3.775793] random: systemd: uninitialized urandom read (16 bytes read) [ 3.781315] systemd[1]: Initializing machine ID from random generator. [ 3.935395] random: ln: uninitialized urandom read (6 bytes read) [ 4.104450] random: systemd: uninitialized urandom read (16 bytes read) [ 4.108252] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 4.114624] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 4.121759] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Swap. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Timers. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Initrd Root Device. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Journal Service. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 6.050975] device-mapper: uevent: version 1.0.3 [ 6.056259] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 8.244921] virtio_net virtio0 ens2: renamed from eth0 Starting dracut initqueue hook... [ 8.417352] scsi host0: ata_piix [ 8.502274] scsi host1: ata_piix [ 8.512794] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 8.525830] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 13.084571] random: crng init done [ 13.087932] random: 7 urandom warning(s) missed due to ratelimiting [ 13.481228] dracut-initqueue[585]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ 16.248883] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Reached target Remote File Systems. [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 19.946232] printk: systemd: 22 output lines suppressed due to ratelimiting [ 21.053125] SELinux: Disabled at runtime. [ 21.222505] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 21.254368] systemd[1]: Detected virtualization kvm. [ 21.265371] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 23.192568] systemd[1]: initrd-switch-root.service: Succeeded. [ 23.198474] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 23.221177] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 23.228882] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 23.235144] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 23.261377] systemd[1]: Starting Journal Service... Starting Journal Service... [ 23.281507] systemd[1]: proc-sys-fs-binfmt_misc.automount: Refusing to start, unit to trigger not loaded. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Remount Root and Kernel File Systems... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-getty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. Mounting Kernel Debug File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on Process Core Dump Socket. Starting udev Coldplug all Devices... Starting Apply Kernel Variables... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Huge Pages File System... [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target Paths. [ 23.910621] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Create list of required st…ce nodes for the current kernel... Mounting POSIX Message Queue File System... [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 25.125805] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 26.661638] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 26.704279] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 27.157837] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 27.537351] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit)[ 31.558775] Key type dns_resolver registered [** ] A start job is running for Configur…-only root support (8s / no limit)[ 32.483296] NFS: Registering the id_resolver key type [ 32.485178] Key type id_resolver registered [ 32.487674] Key type id_legacy registered [*** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Login Service... Starting Network Manager... [ OK ] Started Daily Cleanup of Temporary Directories. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg117-client login: [ 52.373346] hrtimer: interrupt took 4951867 ns [ 94.150067] libcfs: loading out-of-tree module taints kernel. [ 94.274558] Key type ._llcrypt registered [ 94.282481] Key type .llcrypt registered [ 95.081505] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 95.106956] alg: No test for adler32 (adler32-zlib) [ 96.766080] Lustre: Lustre: Build Version: 2.17.58_39_ge07ba0b [ 97.404401] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 99.135243] Key type lgssc registered [ 101.514703] Lustre: Echo OBD driver; http://www.lustre.org/ [ 232.473641] Lustre: Mounted lustre-client - version 2.17.58_39_ge07ba0b [ 237.326717] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 251.501034] Lustre: DEBUG MARKER: oleg117-client.virtnet: executing check_logdir /tmp/testlogs/ [ 257.086646] Lustre: DEBUG MARKER: oleg117-client.virtnet: executing yml_node [ 258.015651] Lustre: lustre-OST0000-osc-ffffa00504cb5000: disconnect after 23s idle [ 263.265682] Lustre: DEBUG MARKER: Client: 2.17.58.39 [ 265.409552] Lustre: DEBUG MARKER: MDS: 2.17.58.39 [ 268.533842] Lustre: DEBUG MARKER: OSS: 2.17.58.39 [ 270.261646] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Tue Sep 15 00:14:15 EDT 2026 [ 286.948679] Lustre: Unmounted lustre-client [ 310.814759] Key type lgssc unregistered [ 311.044293] LNet: 8420:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 311.054164] LNetError: 8420:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 311.074977] LNet: Removed LNI 192.168.201.17@tcp [ 311.675234] Key type .llcrypt unregistered [ 311.678958] Key type ._llcrypt unregistered [ 337.487387] Key type ._llcrypt registered [ 337.501944] Key type .llcrypt registered [ 337.707952] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 337.725177] alg: No test for adler32 (adler32-zlib) [ 338.595906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 338.650829] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 338.676450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show [ 342.310729] Lustre: DEBUG MARKER: excepting tests: 253 254 410 [ 342.548071] LNet: 9297:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 342.565342] LNetError: 9297:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 342.588688] LNet: Removed LNI 192.168.201.17@tcp [ 343.177587] Key type .llcrypt unregistered [ 343.180340] Key type ._llcrypt unregistered [ 349.254567] Key type ._llcrypt registered [ 349.256868] Key type .llcrypt registered [ 350.244171] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 00:15:35 (1789445735) [ 350.627894] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 350.638253] alg: No test for adler32 (adler32-zlib) [ 351.503751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 351.554440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 351.571876] LNet: 9707:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 351.582885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 351.610228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 351.620260] LNet: 9714:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 357.323891] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 00:15:43 (1789445743) [ 357.464182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 357.512343] LNet: 10288:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 357.539561] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 357.620154] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp [ 363.723943] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 00:15:49 (1789445749) [ 363.834534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 363.896056] LNet: 10874:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 363.968931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 364.100586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 2.2.2.2@o2ib [ 369.799772] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 00:15:55 (1789445755) [ 369.916234] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 369.937277] LNet: 11460:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 369.968476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 370.071423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib [ 376.251405] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 00:16:02 (1789445762) [ 376.493086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 376.512214] LNet: 12050:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 376.525610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 376.583084] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp [ 376.897729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp [ 382.104335] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 00:16:07 (1789445767) [ 382.214395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 382.246426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 382.322758] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 382.661956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 383.151316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib [ 388.580899] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 00:16:14 (1789445774) [ 388.646373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 388.679753] LNet: 13228:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 388.686666] LNet: 13228:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 388.707411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 388.763392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi,[6-12/4]@bxi3f [ 394.402514] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 00:16:20 (1789445780) [ 394.505251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 394.553755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 394.654894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp [ 394.685333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 395.073907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib [ 395.103492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 395.414219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp [ 395.462462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 395.705749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib [ 395.731955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 396.095662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib [ 396.137519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 396.637741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni [ 396.676705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 397.186850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni [ 397.235344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 397.558402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi [ 397.591447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 397.912347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi [ 397.938676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 398.235903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f [ 398.271873] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 398.573707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f --nid [8-12]@bxi3f [ 398.607708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 398.997033] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi,[3-7]@bxi3f [ 399.044798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 399.479491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 399.523664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 405.508425] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 00:16:31 (1789445791) [ 405.732973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 405.772143] LNet: 14501:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 405.779699] LNet: 14501:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 405.824382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 405.932262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp [ 405.974647] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp [ 412.326087] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 00:16:37 (1789445797) [ 412.432181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 412.508959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 412.596131] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 412.656448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 418.792417] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 00:16:44 (1789445804) [ 418.860721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 418.892633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 418.968398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp [ 419.000675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp [ 423.954346] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 00:16:49 (1789445809) [ 424.203810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 424.274727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 424.358629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 424.385773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 430.472271] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 00:16:55 (1789445815) [ 430.785340] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 430.840950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 430.949454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp [ 430.986701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp [ 436.466663] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 00:17:02 (1789445822) [ 436.631890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 436.721698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 436.821366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 436.864126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 442.115652] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 00:17:07 (1789445827) [ 442.185449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 442.213977] LNet: 18012:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 442.219378] LNet: 18012:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 442.256192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 442.338155] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 442.704160] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 448.396426] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 00:17:14 (1789445834) [ 448.452533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 448.498708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 448.681720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 448.866388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 454.123827] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 00:17:19 (1789445839) [ 454.363681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 454.414655] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 454.575286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 454.775561] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 460.418519] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 00:17:26 (1789445846) [ 460.534693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 460.587840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 460.974504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 461.177924] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 466.621749] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 00:17:32 (1789445852) [ 467.001473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 467.044773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 467.129577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 467.173079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 472.919741] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 00:17:38 (1789445858) [ 473.376827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 473.442455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 473.566885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 473.621028] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 480.219334] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 00:17:45 (1789445865) [ 480.294994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 480.320117] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 480.437517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 19@gni [ 486.543209] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 00:17:52 (1789445872) [ 486.655537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 486.701740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 486.779634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib [ 492.631657] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 00:17:58 (1789445878) [ 492.856494] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 492.920707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 492.984616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni [ 493.527709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni [ 500.089421] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 00:18:05 (1789445885) [ 500.190147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 500.248109] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 500.353062] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni [ 500.394397] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 22@gni --nid 26@gni [ 506.913317] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 00:18:12 (1789445892) [ 507.043313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 507.063227] LNet: 24041:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 507.068785] LNet: 24041:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 9 previous similar messages [ 507.111759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 507.179687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni [ 507.214969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni [ 513.475749] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 00:18:18 (1789445898) [ 513.595959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 513.643323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 513.732701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni [ 513.798279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp [ 520.612820] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 00:18:26 (1789445906) [ 520.885115] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 520.939791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 521.016584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 521.063386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 528.213521] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 00:18:33 (1789445913) [ 528.343248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 528.415369] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 528.550657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim [ 528.674965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp [ 528.707893] LNetError: 25814:0:(peer.c:2366:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 528.747900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --force [ 528.774958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show --nid 1.1.1.1@tcp [ 536.512250] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 00:18:41 (1789445921) [ 536.652802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 536.722846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 536.878901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.E9r4uCo1yI [ 536.958239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.E9r4uCo1yI [ 537.007671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.E9r4uCo1yI [ 544.288358] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 00:18:49 (1789445929) [ 544.594939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 544.640444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 544.701153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp [ 544.728644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp [ 544.789777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer list [ 551.992838] Lustre: DEBUG MARKER: == sanity-lnet test 40: LNetAddPeer merges a pre-existing non-primary peer NID ========================================================== 00:18:57 (1789445937) [ 552.061821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 552.123849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 552.256365] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 552.288800] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 554.385058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 556.115241] Lustre: Lustre: Build Version: 2.17.58_39_ge07ba0b [ 558.207339] Key type lgssc registered [ 559.891802] Lustre: Echo OBD driver; http://www.lustre.org/ [ 564.409888] LNetError: 27574:0:(lib-move.c:2151:lnet_handle_find_routed_path()) peer 192.168.201.17@tcp99 has no available nets [ 567.661496] Key type lgssc unregistered [ 568.105527] LNetError: 28680:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 568.126374] LNet: Removed LNI 192.168.201.17@tcp [ 569.106104] Key type .llcrypt unregistered [ 569.119502] Key type ._llcrypt unregistered [ 576.783529] Key type ._llcrypt registered [ 576.794414] Key type .llcrypt registered [ 578.438505] Lustre: DEBUG MARKER: == sanity-lnet test 50: Enable/disable routing via yaml import ========================================================== 00:19:23 (1789445963) [ 578.967715] Key type .llcrypt unregistered [ 578.971786] Key type ._llcrypt unregistered [ 585.610672] Key type ._llcrypt registered [ 585.617652] Key type .llcrypt registered [ 585.923834] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 585.956776] alg: No test for adler32 (adler32-zlib) [ 586.855572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 586.906669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 586.928740] LNet: 29588:0:(router.c:1481:lnet_nrb_tiny_calculate()) tiny_router_buffers=16 less than recommended minimum 512 [ 586.933657] LNet: 29588:0:(router.c:1503:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096 [ 586.938398] LNet: 29588:0:(router.c:1525:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256 [ 586.949886] LNet: Message forwarding will be enabled in 2s [ 590.048582] LNet: Message forwarding enabled [ 590.443400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 590.465320] LNet: Message forwarding will stop in 2s to 6s [ 594.157963] LNet: Message forwarding disabled [ 594.954737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 594.975449] LNet: 29686:0:(router.c:1481:lnet_nrb_tiny_calculate()) tiny_router_buffers=16 less than recommended minimum 512 [ 594.980646] LNet: 29686:0:(router.c:1503:lnet_nrb_small_calculate()) small_router_buffers=8 less than recommended minimum 4096 [ 594.993276] LNet: 29686:0:(router.c:1525:lnet_nrb_large_calculate()) large_router_buffers=4 less than recommended minimum 256 [ 595.010583] LNet: Message forwarding will be enabled in 2s [ 595.184457] LNet: 29699:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 595.744880] Key type .llcrypt unregistered [ 595.747559] Key type ._llcrypt unregistered [ 608.100935] Key type ._llcrypt registered [ 608.104285] Key type .llcrypt registered [ 609.296600] Lustre: DEBUG MARKER: == sanity-lnet test 51: Set numa range via yaml import === 00:19:54 (1789445994) [ 609.914529] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 609.929871] alg: No test for adler32 (adler32-zlib) [ 610.910639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 610.970757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-51.yaml [ 616.822981] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 00:20:02 (1789446002) [ 617.095142] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 617.121440] LNet: 31099:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 617.150561] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 617.220947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid foobar [ 617.253744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid foobar [ 617.278452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@o2ib [ 617.308432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --nid 1.1.1.1@tcp [ 617.333849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --nid 1.1.1.1@tcp [ 617.399210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add [ 617.475304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 617.522472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp [ 617.560110] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@foo [ 617.597470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@tcp [ 617.625788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@o2ib [ 617.668078] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@gni [ 617.720814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni [ 617.770456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni [ 617.801954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp [ 617.845139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp [ 617.870265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp [ 617.926272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 617.967072] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp --gateway 1@gni [ 617.999902] LNetError: 31142:0:(router.c:625:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni [ 625.125724] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 00:20:09 (1789446009) [ 625.318194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 625.355965] LNet: 31708:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 625.375832] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 625.477577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 633.887862] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 00:20:19 (1789446019) [ 636.534099] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_100 Need at least 2 remote nodes found oleg117-server [ 638.695388] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 00:20:23 (1789446023) [ 638.867968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 638.951529] LNet: 32638:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 639.011049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 639.163538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 639.191622] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 640.345220] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp1 --gateway 192.168.201.[18-22]@tcp [ 640.366829] LNet: 32777:0:(router.c:729:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 640.430268] LNetError: 32648:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.18@tcp is being used as a gateway but routing feature is not turned on [ 640.463143] LNetError: 32648:0:(lib-lnet.h:1281:lnet_set_route_aliveness()) route to tcp1 through 192.168.201.18@tcp has gone from up to down [ 649.209750] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 00:20:34 (1789446034) [ 651.180483] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_102 Need at least 2 remote nodes found oleg117-server [ 653.018679] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 00:20:38 (1789446038) [ 653.244060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 653.270470] LNet: 33709:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 653.278218] LNetError: 33709:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 654.322693] LNet: Removed LNI 192.168.201.17@tcp [ 654.395659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 654.501231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 654.536195] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 655.410349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp103 --gateway 192.168.201.[18-23/2]@tcp [ 655.438519] LNet: 33830:0:(router.c:729:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 655.454683] LNet: 33830:0:(router.c:729:lnet_add_route()) Skipped 4 previous similar messages [ 655.480731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route del --net tcp103 --gateway 192.168.201.[18-23/2]@tcp [ 663.495849] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 00:20:49 (1789446049) [ 664.006328] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 664.054929] LNet: 34423:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 664.063518] LNetError: 34423:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 664.084283] LNet: Removed LNI 192.168.201.17@tcp [ 664.122394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 664.289425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking -1 [ 664.417137] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 664.525857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 664.670995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 664.822822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 664.917555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 665.019523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 0 [ 665.071382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 665.102515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 665.167060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 665.259850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 665.293728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 665.376622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 1 [ 665.466603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 665.516708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 665.603572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 665.703361] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 665.751565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 665.810634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 2 [ 665.890768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 665.938372] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 666.044940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 666.212031] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 666.305170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 666.369243] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 666.565952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 666.648362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 666.790816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 666.941917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 667.017776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 667.116538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 4 [ 667.207883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 667.269408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 667.408828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 673.898401] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 00:20:59 (1789446059) [ 676.184373] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_105 Need at least 2 remote nodes found oleg117-server [ 678.749375] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 00:21:04 (1789446064) [ 680.964161] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_106 Need at least 2 remote nodes found oleg117-server [ 683.106316] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 00:21:08 (1789446068) [ 683.316255] LNet: 35820:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 683.322244] LNet: 35820:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 11 previous similar messages [ 683.960392] Key type .llcrypt unregistered [ 683.962105] Key type ._llcrypt unregistered [ 692.793019] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 693.089867] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 693.107749] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 693.114460] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 696.318807] Key type ._llcrypt registered [ 696.320779] Key type .llcrypt registered [ 696.701837] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 696.737168] alg: No test for adler32 (adler32-zlib) [ 697.811985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 697.912919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 697.983891] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 699.262439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 699.294958] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 702.846581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 702.881710] LNet: Removed LNI 10.1.2.3@tcp [ 703.101090] LNet: 36546:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 703.117527] LNetError: 36546:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 703.133035] LNet: Removed LNI 192.168.201.17@tcp [ 704.059587] Key type .llcrypt unregistered [ 704.080791] Key type ._llcrypt unregistered [ 720.032371] Key type ._llcrypt registered [ 720.035243] Key type .llcrypt registered [ 721.973571] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 00:21:46 (1789446106) [ 722.466508] Key type .llcrypt unregistered [ 722.472495] Key type ._llcrypt unregistered [ 728.138073] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 728.336105] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 728.364186] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 728.373614] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 731.085058] Key type ._llcrypt registered [ 731.092358] Key type .llcrypt registered [ 731.378859] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 731.406409] alg: No test for adler32 (adler32-zlib) [ 732.412218] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 732.474122] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 732.492204] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 733.109851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 733.132915] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 735.235810] LNet: 38035:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 735.238679] LNetError: 38035:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 735.242705] LNet: Removed LNI 10.1.2.3@tcp [ 735.251750] LNet: Removed LNI 192.168.201.17@tcp [ 735.753451] Key type .llcrypt unregistered [ 735.755062] Key type ._llcrypt unregistered [ 746.687390] Key type ._llcrypt registered [ 746.689718] Key type .llcrypt registered [ 747.925833] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 00:22:13 (1789446133) [ 748.829529] Key type .llcrypt unregistered [ 748.832119] Key type ._llcrypt unregistered [ 754.464534] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 754.562219] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 754.569385] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 754.573729] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 757.148954] Key type ._llcrypt registered [ 757.151570] Key type .llcrypt registered [ 757.289790] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 757.307299] alg: No test for adler32 (adler32-zlib) [ 758.215050] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 758.248878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 758.267325] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 759.927425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 759.957212] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 761.461532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 761.486844] LNet: Removed LNI 10.1.2.3@tcp [ 761.504179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg:0 [ 761.530609] LNet: Removed LNI 10.1.2.31@tcp [ 761.546868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 761.573340] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 762.357355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 762.372312] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 763.289654] LNet: 39948:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 763.294306] LNetError: 39948:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg:0 not found [ 763.299505] LNet: Removed LNI 10.1.2.31@tcp [ 763.307614] LNet: Removed LNI 10.1.2.3@tcp [ 763.822622] Key type .llcrypt unregistered [ 763.824825] Key type ._llcrypt unregistered [ 776.501516] Key type ._llcrypt registered [ 776.503581] Key type .llcrypt registered [ 777.900966] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 00:22:43 (1789446163) [ 778.448862] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 778.460994] alg: No test for adler32 (adler32-zlib) [ 779.492626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 779.545656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 779.565531] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 780.200946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 780.220234] LNet: 40897:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 780.232485] LNetError: 40897:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 780.244903] LNet: Removed LNI 192.168.201.17@tcp [ 780.291632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 780.362323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --nid 192.168.201.17@tcp [ 780.391542] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 781.038516] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --nid 192.168.201.17@tcp [ 781.063764] LNet: Removed LNI 192.168.201.17@tcp [ 781.120357] LNet: 41032:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 781.635205] Key type .llcrypt unregistered [ 781.637930] Key type ._llcrypt unregistered [ 793.470828] Key type ._llcrypt registered [ 793.472725] Key type .llcrypt registered [ 794.426951] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 00:23:00 (1789446180) [ 796.257482] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_111 Need at least 2 remote nodes found oleg117-server [ 797.849699] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 00:23:03 (1789446183) [ 798.309697] Key type .llcrypt unregistered [ 798.311107] Key type ._llcrypt unregistered [ 803.706806] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 804.016568] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 804.021345] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 804.038817] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 806.587154] Key type ._llcrypt registered [ 806.591786] Key type .llcrypt registered [ 806.824296] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 806.832912] alg: No test for adler32 (adler32-zlib) [ 807.853982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 807.924175] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 808.472765] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180] [ 810.158170] LNet: 42863:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 810.166273] LNetError: 42863:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 810.170520] LNet: Removed LNI 192.168.201.17@tcp1 [ 810.179312] LNet: Removed LNI 10.1.2.3@tcp2 [ 810.813184] Key type .llcrypt unregistered [ 810.816678] Key type ._llcrypt unregistered [ 821.675730] Key type ._llcrypt registered [ 821.677921] Key type .llcrypt registered [ 823.178949] Lustre: DEBUG MARKER: == sanity-lnet test 150: Check import of ip2nets yaml sequence ========================================================== 00:23:28 (1789446208) [ 823.579229] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 823.593781] alg: No test for adler32 (adler32-zlib) [ 824.577477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 824.618174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 824.632693] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 825.271951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 825.286463] LNet: 43829:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 825.294626] LNetError: 43829:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 825.309943] LNet: Removed LNI 192.168.201.17@tcp [ 825.322378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 825.368648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet.yaml [ 825.386985] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 831.489905] Lustre: DEBUG MARKER: == sanity-lnet test 151: Check correct application of multiple ip2nets rules ========================================================== 00:23:37 (1789446217) [ 831.586681] LNet: 44411:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 831.599506] LNetError: 44411:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 831.631271] LNet: Removed LNI 192.168.201.17@tcp [ 832.112581] Key type .llcrypt unregistered [ 832.115175] Key type ._llcrypt unregistered [ 838.455217] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 838.705742] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 838.713610] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 838.729228] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 841.261238] Key type ._llcrypt registered [ 841.264360] Key type .llcrypt registered [ 841.490395] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 841.517135] alg: No test for adler32 (adler32-zlib) [ 842.441324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 842.505269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 842.530226] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 843.217990] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 843.245114] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 845.750825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 845.787435] LNet: 45160:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 845.798788] LNetError: 45160:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 845.803157] LNet: Removed LNI 10.1.2.3@tcp [ 845.823157] LNet: Removed LNI 192.168.201.17@tcp [ 845.853266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 845.944330] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 845.978827] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 845.991813] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 846.443273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 846.470945] LNet: 45187:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 846.483499] LNetError: 45187:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 846.493117] LNetError: 45187:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 846.499986] LNet: Removed LNI 10.1.2.3@tcp [ 846.519892] LNet: Removed LNI 192.168.201.17@tcp [ 846.565855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 846.631929] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 846.650750] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 846.654339] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 847.043296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 847.067840] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 848.259389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 848.281625] LNet: 45349:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 848.305083] LNetError: 45349:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 848.321726] LNetError: 45349:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 848.332605] LNet: Removed LNI 10.1.2.3@tcp [ 848.338465] LNet: Removed LNI 192.168.201.17@tcp [ 848.352759] LNet: Removed LNI 192.168.201.17@tcp2 [ 848.380959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 848.446202] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 848.469886] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 848.473705] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 848.479046] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 855.659457] Lustre: DEBUG MARKER: == sanity-lnet test 152: Check import of ip2nets with wildcard IP patterns ========================================================== 00:24:01 (1789446241) [ 855.759809] LNet: 45940:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 855.766610] LNetError: 45940:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 855.776353] LNetError: 45940:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 855.781600] LNet: Removed LNI 10.1.2.3@tcp [ 855.791214] LNet: Removed LNI 192.168.201.17@tcp [ 855.805608] LNet: Removed LNI 192.168.201.17@tcp2 [ 856.340381] Key type .llcrypt unregistered [ 856.345990] Key type ._llcrypt unregistered [ 862.411839] Key type ._llcrypt registered [ 862.420890] Key type .llcrypt registered [ 862.729872] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 862.765893] alg: No test for adler32 (adler32-zlib) [ 863.753971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 863.830848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 863.854536] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 864.744184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 864.763878] LNet: 46339:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 864.769426] LNetError: 46339:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 864.811488] LNet: Removed LNI 192.168.201.17@tcp [ 864.860837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 865.001915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 865.066258] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 865.748357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 865.796294] LNet: 46369:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 865.801892] LNetError: 46369:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 865.827688] LNet: Removed LNI 192.168.201.17@tcp [ 865.846805] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 865.951955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 865.979287] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 866.492179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 866.520807] LNet: Removed LNI 192.168.201.17@tcp [ 866.563931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 866.682363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 866.716715] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 867.332460] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 867.495768] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 867.501244] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 867.510394] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 869.889651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 869.909647] LNet: 46472:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 869.916527] LNet: 46472:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 869.924826] LNetError: 46472:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 869.930851] LNetError: 46472:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 869.942674] LNet: Removed LNI 192.168.201.17@tcp [ 869.970988] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 870.050586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 870.071242] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 875.969094] Lustre: DEBUG MARKER: == sanity-lnet test 153: Check import of ip2nets with IP address ranges ========================================================== 00:24:21 (1789446261) [ 876.117852] LNet: 47064:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 876.140662] LNetError: 47064:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 876.149990] LNet: Removed LNI 192.168.201.17@tcp [ 876.639304] Key type .llcrypt unregistered [ 876.641235] Key type ._llcrypt unregistered [ 881.848137] Key type ._llcrypt registered [ 881.850319] Key type .llcrypt registered [ 882.093433] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 882.113530] alg: No test for adler32 (adler32-zlib) [ 882.987883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 883.040790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 883.069840] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 883.813187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 883.839931] LNet: 47465:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 883.845952] LNetError: 47465:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 883.865832] LNet: Removed LNI 192.168.201.17@tcp [ 883.885579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 883.937258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-153.yaml [ 883.968200] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 890.261139] Lustre: DEBUG MARKER: == sanity-lnet test 154: Check import of ip2nets with stepped IP address ranges ========================================================== 00:24:35 (1789446275) [ 890.380992] LNet: 48049:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 890.400881] LNetError: 48049:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 890.432121] LNet: Removed LNI 192.168.201.17@tcp [ 890.963411] Key type .llcrypt unregistered [ 890.964945] Key type ._llcrypt unregistered [ 897.173591] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 897.316873] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 897.324162] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 897.339833] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 900.014856] Key type ._llcrypt registered [ 900.017952] Key type .llcrypt registered [ 900.304856] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 900.317494] alg: No test for adler32 (adler32-zlib) [ 901.327825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 901.375451] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 901.394331] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 902.129738] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 902.152235] LNet: 48504:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 902.157542] LNetError: 48504:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 902.178824] LNet: Removed LNI 192.168.201.17@tcp [ 902.196173] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 902.262548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-154.yaml [ 902.292120] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 908.920811] Lustre: DEBUG MARKER: == sanity-lnet test 155: Check ip2nets import failure with no matching rules ========================================================== 00:24:54 (1789446294) [ 909.119082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 909.162271] LNet: 49104:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 909.185106] LNetError: 49104:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 909.218120] LNet: Removed LNI 192.168.201.17@tcp [ 909.249733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 909.391406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-155.yaml [ 916.499815] Lustre: DEBUG MARKER: == sanity-lnet test 156: Check import of ip2nets with comma-separated IP ranges ========================================================== 00:25:01 (1789446301) [ 916.782020] LNet: 49687:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 917.524807] Key type .llcrypt unregistered [ 917.528422] Key type ._llcrypt unregistered [ 924.235525] Key type ._llcrypt registered [ 924.237592] Key type .llcrypt registered [ 924.413613] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 924.433826] alg: No test for adler32 (adler32-zlib) [ 925.352942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 925.418979] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 925.456948] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 926.281540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 926.298729] LNet: 50081:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 926.303601] LNetError: 50081:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 926.317758] LNet: Removed LNI 192.168.201.17@tcp [ 926.355403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 926.461967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-156.yaml [ 926.484331] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 932.582670] Lustre: DEBUG MARKER: == sanity-lnet test 157: Check import failure with malformed ip2nets YAML ========================================================== 00:25:18 (1789446318) [ 932.958469] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 932.977305] LNet: 50674:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 932.982640] LNetError: 50674:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 932.996474] LNet: Removed LNI 192.168.201.17@tcp [ 933.014792] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 933.083653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-157.yaml [ 939.237130] Lustre: DEBUG MARKER: == sanity-lnet test 158: Check import of ip2nets with single IPv6 address ========================================================== 00:25:24 (1789446324) [ 940.644095] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_158 Interface has no IPv6 [ 942.113600] Lustre: DEBUG MARKER: == sanity-lnet test 159: Check import of ip2nets with IPv6 /128 CIDR notation ========================================================== 00:25:27 (1789446327) [ 943.510979] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_159 Interface has no IPv6 [ 945.146538] Lustre: DEBUG MARKER: == sanity-lnet test 160: Check correct application of multiple IPv6 ip2nets rules ========================================================== 00:25:30 (1789446330) [ 946.799177] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_160 Interface has no IPv6 [ 948.348761] Lustre: DEBUG MARKER: == sanity-lnet test 161: Check import of ip2nets with various IPv6 prefix lengths ========================================================== 00:25:33 (1789446333) [ 949.870861] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_161 Interface has no IPv6 [ 951.569551] Lustre: DEBUG MARKER: == sanity-lnet test 162: Check ip2nets import failure with non-matching IPv6 ========================================================== 00:25:37 (1789446337) [ 952.989529] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_162 Interface has no IPv6 [ 954.815442] Lustre: DEBUG MARKER: == sanity-lnet test 170: Check CPTs and tunables parsing when importing ========================================================== 00:25:40 (1789446340) [ 954.963549] LNet: 52689:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 955.749338] Key type .llcrypt unregistered [ 955.753277] Key type ._llcrypt unregistered [ 962.492792] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 962.773116] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 962.792413] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 962.796173] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 965.593230] Key type ._llcrypt registered [ 965.601168] Key type .llcrypt registered [ 965.939306] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 965.976353] alg: No test for adler32 (adler32-zlib) [ 966.947209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 967.075654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 967.136632] LNet: Added LNI 192.168.201.17@tcp [8/512/0/180] [ 967.147914] LNet: Added LNI 10.1.2.3@tcp [8/512/0/180] [ 967.157986] LNet: 53014:0:(api-ni.c:2991:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 967.371752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 967.395793] LNet: 53033:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 967.404494] LNetError: 53033:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 967.412803] LNet: Removed LNI 10.1.2.3@tcp [ 967.418623] LNetError: 53033:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 967.429311] LNet: Removed LNI 192.168.201.17@tcp [ 967.482610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 967.609928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 967.677719] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 967.782972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 967.815965] LNet: 53058:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 967.821936] LNetError: 53058:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 967.834186] LNet: Removed LNI 192.168.201.17@tcp [ 967.862750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 967.935793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 967.974451] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 967.980195] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 967.983926] LNet: 53066:0:(api-ni.c:2991:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 968.166620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 968.205033] LNet: 53084:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 968.215427] LNetError: 53084:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 968.223988] LNet: Removed LNI 10.1.2.3@tcp [ 968.236501] LNetError: 53084:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 968.266212] LNet: Removed LNI 192.168.201.17@tcp [ 968.302891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 968.451978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 968.478298] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 968.486813] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 968.489755] LNet: 53092:0:(api-ni.c:2991:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 968.816215] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 968.847606] LNet: 53118:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 968.862277] LNetError: 53118:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 968.884146] LNet: Removed LNI 10.1.2.3@tcp [ 968.885912] LNetError: 53118:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 968.916174] LNet: Removed LNI 192.168.201.17@tcp [ 968.963533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 969.069290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 969.108556] LNet: Added LNI 192.168.201.17@tcp [8/256/0/182] [ 969.137723] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 969.327925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 969.377793] LNet: 53146:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 969.390220] LNetError: 53146:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 969.402152] LNet: Removed LNI 10.1.2.3@tcp [ 969.407557] LNetError: 53146:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 969.434692] LNet: Removed LNI 192.168.201.17@tcp [ 969.476446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 969.565922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 969.604916] LNet: Added LNI 192.168.201.17@tcp [8/256/0/182] [ 969.623204] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 969.641938] LNet: 53154:0:(api-ni.c:2991:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 969.749902] LNet: Removed LNI 10.1.2.3@tcp [ 969.759067] LNet: Removed LNI 192.168.201.17@tcp [ 970.632528] Key type .llcrypt unregistered [ 970.635673] Key type ._llcrypt unregistered [ 983.760779] Key type ._llcrypt registered [ 983.762870] Key type .llcrypt registered [ 984.957521] Lustre: DEBUG MARKER: == sanity-lnet test 180: FastReg map error should not leak descriptors ========================================================== 00:26:10 (1789446370) [ 986.662396] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_180 Need o2ib NETTYPE [ 989.194599] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 00:26:14 (1789446374) [ 989.654497] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 989.672166] alg: No test for adler32 (adler32-zlib) [ 990.630541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 990.676412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 990.699310] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 991.411146] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --ip2net tcp 192.168.201.17 [ 991.450373] LNet: 54417:0:(config.c:1740:lnet_parse_ip2nets()) Kernel parsing of ip2nets is deprecated. Consider converting to LNet YAML configuration. [ 991.465657] LNet: Removed LNI 192.168.201.17@tcp [ 997.154948] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 00:26:22 (1789446382) [ 997.628850] LNet: 54988:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 997.632960] LNetError: 54988:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 997.651579] LNet: Removed LNI 192.168.201.17@tcp [ 998.274505] Key type .llcrypt unregistered [ 998.276432] Key type ._llcrypt unregistered [ 1004.824420] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1005.142823] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1006.747576] Key type ._llcrypt registered [ 1006.750190] Key type .llcrypt registered [ 1006.917587] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1006.934697] alg: No test for adler32 (adler32-zlib) [ 1008.359339] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1014.380842] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 00:26:40 (1789446400) [ 1014.506105] LNet: 55903:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1014.512720] LNetError: 55903:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1014.538048] LNet: Removed LNI 10.1.2.3@tcp [ 1015.113827] Key type .llcrypt unregistered [ 1015.118774] Key type ._llcrypt unregistered [ 1021.499905] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1021.852838] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1021.856415] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1021.859600] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1024.674671] Key type ._llcrypt registered [ 1024.679285] Key type .llcrypt registered [ 1024.884385] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1024.912347] alg: No test for adler32 (adler32-zlib) [ 1025.910457] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1032.029473] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 00:26:57 (1789446417) [ 1034.191885] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_202 kernel ip2nets deprecated in 2.17.53 [ 1035.666104] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 00:27:01 (1789446421) [ 1035.914552] LNet: 57088:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1035.929372] LNetError: 57088:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1035.945531] LNet: Removed LNI 10.1.2.3@tcp [ 1036.639139] Key type .llcrypt unregistered [ 1036.642183] Key type ._llcrypt unregistered [ 1043.312734] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1043.680809] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1043.687251] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1043.694681] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1046.609268] Key type ._llcrypt registered [ 1046.612495] Key type .llcrypt registered [ 1046.956824] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1046.969642] alg: No test for adler32 (adler32-zlib) [ 1047.905087] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1047.990274] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1056.686741] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 00:27:22 (1789446442) [ 1057.012516] LNet: 58315:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1057.016989] LNetError: 58315:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1057.029399] LNet: Removed LNI 10.1.2.3@tcp [ 1057.527419] Key type .llcrypt unregistered [ 1057.529307] Key type ._llcrypt unregistered [ 1063.528987] Key type ._llcrypt registered [ 1063.536163] Key type .llcrypt registered [ 1063.796457] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1063.824633] alg: No test for adler32 (adler32-zlib) [ 1064.821903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1064.853227] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1073.335539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1073.997395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_interrupt -r 1 [ 1074.037985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_interrupt -r 1 [ 1074.114788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1074.139170] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1074.435803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1074.462431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1074.505919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1074.538283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1074.602077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1074.638542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1074.678382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1074.722867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1074.772437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1074.807204] LNet: 58752:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1074.836842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1074.905675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1074.955079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1074.991728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1075.026472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1075.052828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1075.077860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1075.603826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_dropped -r 1 [ 1075.637967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_dropped -r 1 [ 1075.664610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1075.679177] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1075.875860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1075.904179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1075.934583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1075.962775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1075.993504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1076.019474] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1076.046200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1076.086613] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1076.113694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1076.127864] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1076.132410] LNet: 58864:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1076.162376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1076.207843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1076.260371] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1076.290452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1076.329165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1076.361107] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1076.416926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1077.068717] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_aborted -r 1 [ 1077.108454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_aborted -r 1 [ 1077.144574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1077.163452] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1077.169513] LNet: Skipped 1 previous similar message [ 1077.374842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1077.419157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1077.481884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1077.522995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1077.573931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1077.623978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1077.661463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1077.717355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1077.773191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1077.808112] LNet: 58975:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1077.847476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1077.933510] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1077.986658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1078.024630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1078.065282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1078.090925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1078.116500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1078.642634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_no_route -r 1 [ 1078.693446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_no_route -r 1 [ 1078.739763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1078.984832] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1079.009439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1079.029438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1079.052626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1079.078441] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1079.110982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1079.133416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1079.165893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1079.202036] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1079.230566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1079.271428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1079.321451] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1079.371464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1079.418088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1079.459727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1079.508913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1080.025416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_timeout -r 1 [ 1080.055957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_timeout -r 1 [ 1080.091178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1080.110402] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1080.114507] LNet: Skipped 1 previous similar message [ 1080.330513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1080.376626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1080.467411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1080.493463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1080.526364] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1080.553690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1080.584875] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1080.608739] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1080.632579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1080.649455] LNet: 59198:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1080.655413] LNet: 59198:0:(api-ni.c:301:recovery_interval_set()) Skipped 1 previous similar message [ 1080.672177] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1080.705360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1080.737419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1080.764613] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1080.802327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1080.855183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1080.902032] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1081.554599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_error -r 1 [ 1081.585092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_error -r 1 [ 1081.616594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1081.885325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1081.912351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1081.941520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1081.971677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1081.995777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1082.016331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1082.042575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1082.072866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1082.102709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1082.137622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1082.189130] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1082.231584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1082.266118] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1082.325846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1082.402663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1082.459876] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1082.740634] LNet: 59350:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1082.746943] LNetError: 59350:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1083.825919] LNet: Removed LNI 192.168.201.17@tcp [ 1084.560558] Key type .llcrypt unregistered [ 1084.562789] Key type ._llcrypt unregistered [ 1102.199337] Key type ._llcrypt registered [ 1102.201638] Key type .llcrypt registered [ 1103.602364] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 00:28:09 (1789446489) [ 1104.485445] Key type .llcrypt unregistered [ 1104.487610] Key type ._llcrypt unregistered [ 1110.943443] Key type ._llcrypt registered [ 1110.946508] Key type .llcrypt registered [ 1111.202259] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1111.238168] alg: No test for adler32 (adler32-zlib) [ 1112.151778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1112.175913] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1120.289553] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1131.151969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1131.211471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1131.258662] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1132.954966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_interrupt -r 1 [ 1133.039193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_interrupt -r 1 [ 1133.088925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_interrupt -r 1 [ 1133.145292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_interrupt -r 1 [ 1133.180580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_interrupt -r 1 [ 1133.228788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_interrupt -r 1 [ 1133.265441] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_interrupt -r 1 [ 1133.301926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_interrupt -r 1 [ 1133.350462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1133.372375] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1133.378732] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1133.637643] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1133.724074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1133.796191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1133.856660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1133.899438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1133.972582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1134.026084] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1134.086864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1134.127491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1134.166942] LNet: 60817:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1134.205495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1134.252634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1134.303490] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1134.350317] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1134.427923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1134.432797] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1134.452653] LNet: Skipped 3 previous similar messages [ 1134.501297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1134.572963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1135.041296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_dropped -r 1 [ 1135.067484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_dropped -r 1 [ 1135.095464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_dropped -r 1 [ 1135.147915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_dropped -r 1 [ 1135.192446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_dropped -r 1 [ 1135.230580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_dropped -r 1 [ 1135.268359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_dropped -r 1 [ 1135.293901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_dropped -r 1 [ 1135.322135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1135.611574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1135.634428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1135.662883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1135.687695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1135.715310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1135.743206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1135.770741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1135.796980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1135.826666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1135.843542] LNet: 60940:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1135.861911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1135.926605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1135.969557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1136.000961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1136.029626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1136.066727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1136.094532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1136.651153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_aborted -r 1 [ 1136.691922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_aborted -r 1 [ 1136.738224] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_aborted -r 1 [ 1136.792433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_aborted -r 1 [ 1136.863100] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_aborted -r 1 [ 1136.895237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_aborted -r 1 [ 1136.950262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_aborted -r 1 [ 1136.995956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_aborted -r 1 [ 1137.032896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1137.050373] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1137.057151] LNet: Skipped 3 previous similar messages [ 1137.283747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1137.311932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1137.339671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1137.365755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1137.395513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1137.420142] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1137.451227] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1137.484715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1137.531154] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1137.560755] LNet: 61064:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1137.578744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1137.642454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1137.682483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1137.709955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1137.738598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1137.768914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1137.799935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1138.253945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_no_route -r 1 [ 1138.297783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_no_route -r 1 [ 1138.356773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_no_route -r 1 [ 1138.420382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_no_route -r 1 [ 1138.489444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_no_route -r 1 [ 1138.544893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_no_route -r 1 [ 1138.601296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_no_route -r 1 [ 1138.669305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_no_route -r 1 [ 1138.719383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1139.012629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1139.042951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1139.074491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1139.116448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1139.146629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1139.174422] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1139.203632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1139.233836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1139.263238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1139.293847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1139.329848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1139.362489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1139.392818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1139.450707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1139.520664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1139.605307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1140.357200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_timeout -r 1 [ 1140.450390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_timeout -r 1 [ 1140.547171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_timeout -r 1 [ 1140.589531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_timeout -r 1 [ 1140.634951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_timeout -r 1 [ 1140.678826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_timeout -r 1 [ 1140.737447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_timeout -r 1 [ 1140.791446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_timeout -r 1 [ 1140.837827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1140.862092] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1140.866548] LNet: Skipped 5 previous similar messages [ 1141.117595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1141.169916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1141.230290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1141.259363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1141.290490] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1141.326656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1141.379691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1141.418850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1141.454923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1141.491826] LNet: 61310:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1141.511775] LNet: 61310:0:(api-ni.c:301:recovery_interval_set()) Skipped 1 previous similar message [ 1141.567462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1141.654484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1141.729084] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1141.783778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1141.818951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1141.870086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1141.906768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1142.497094] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_error -r 1 [ 1142.556728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_error -r 1 [ 1142.603915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_error -r 1 [ 1142.673476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_error -r 1 [ 1142.734234] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_error -r 1 [ 1142.765039] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_error -r 1 [ 1142.824647] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_error -r 1 [ 1142.884838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_error -r 1 [ 1142.938463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1143.205508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1143.237116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1143.272869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1143.327467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1143.387232] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1143.424959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1143.450456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1143.481919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1143.520763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1143.550273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1143.581270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1143.615745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1143.635673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1143.660643] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1143.694253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1143.725168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1149.084739] LNet: 61510:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1149.095212] LNetError: 61510:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1149.104577] LNet: Removed LNI 192.168.201.17@tcp [ 1150.124724] LNet: Removed LNI 192.168.201.17@tcp1 [ 1150.591621] Key type .llcrypt unregistered [ 1150.594494] Key type ._llcrypt unregistered [ 1166.301738] Key type ._llcrypt registered [ 1166.303234] Key type .llcrypt registered [ 1167.578273] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 00:29:13 (1789446553) [ 1168.263579] Key type .llcrypt unregistered [ 1168.266994] Key type ._llcrypt unregistered [ 1175.885709] Key type ._llcrypt registered [ 1175.893310] Key type .llcrypt registered [ 1176.266475] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1176.314235] alg: No test for adler32 (adler32-zlib) [ 1177.309775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1177.346617] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1186.202076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1187.150548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_dropped -r 1 [ 1187.192994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_dropped -r 1 [ 1187.253637] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1187.294528] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1187.640918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1187.683713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1187.713857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1187.770794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1187.838668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1187.862510] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1187.923516] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1188.008753] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1188.053806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1188.083965] LNet: 62757:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1188.143439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1188.189055] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1188.234634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1188.286492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1188.350823] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1188.409510] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1188.451964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1189.051980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_error -r 1 [ 1189.099908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_error -r 1 [ 1189.145791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1189.173637] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1189.422126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1189.466926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1189.502425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1189.560423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1189.641834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1189.696932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1189.785499] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1189.862068] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1189.957392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1189.989093] LNet: 62869:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1190.040708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1190.177640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1190.244576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1190.302607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1190.390953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1190.494527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1190.603924] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1191.255812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_timeout -r 1 [ 1191.306757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_timeout -r 1 [ 1191.370800] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1191.388112] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1191.785548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1191.824021] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1191.866996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1191.937474] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1192.016802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1192.088755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1192.138946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1192.213932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1192.276736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1192.319417] LNet: 62980:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1192.360431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1192.437092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1192.523139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1192.553111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1192.592840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1192.652877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1192.708590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1193.227919] LNet: 63021:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1193.237665] LNetError: 63021:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1193.275131] LNet: Removed LNI 192.168.201.17@tcp [ 1193.910561] Key type .llcrypt unregistered [ 1193.912660] Key type ._llcrypt unregistered [ 1213.002689] Key type ._llcrypt registered [ 1213.011649] Key type .llcrypt registered [ 1214.187977] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 00:29:59 (1789446599) [ 1214.910579] Key type .llcrypt unregistered [ 1214.915768] Key type ._llcrypt unregistered [ 1220.741596] Key type ._llcrypt registered [ 1220.746555] Key type .llcrypt registered [ 1221.087527] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1221.115522] alg: No test for adler32 (adler32-zlib) [ 1222.081802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1222.099970] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1229.506332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1240.863945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1240.953467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1241.002920] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1242.205369] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_dropped -r 1 [ 1242.236932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e remote_dropped -r 1 [ 1242.278996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_dropped -r 1 [ 1242.319788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e remote_dropped -r 1 [ 1242.361111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e remote_dropped -r 1 [ 1242.408687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e remote_dropped -r 1 [ 1242.440603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e remote_dropped -r 1 [ 1242.496850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e remote_dropped -r 1 [ 1242.522944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1242.539604] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1242.545036] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.117@tcp [ 1242.797724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1242.820848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1242.847937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1242.892679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1242.929226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1242.993508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1243.055382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1243.093845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1243.139626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1243.158489] LNet: 64485:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1243.191752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1243.224645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1243.256312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1243.319274] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1243.378715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1243.410391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1243.450914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1243.490280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1244.049875] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_error -r 1 [ 1244.067387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e remote_error -r 1 [ 1244.087898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_error -r 1 [ 1244.104843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e remote_error -r 1 [ 1244.133437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e remote_error -r 1 [ 1244.167479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e remote_error -r 1 [ 1244.184308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e remote_error -r 1 [ 1244.206517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e remote_error -r 1 [ 1244.225464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1244.235106] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1244.237657] LNet: Skipped 2 previous similar messages [ 1244.396417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1244.431842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1244.454332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1244.477693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1244.502157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1244.530797] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1244.557438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1244.585634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1244.619694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1244.646414] LNet: 64611:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1244.663857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1244.696849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1244.725922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1244.750581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1244.785964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1244.811538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1244.836867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1244.859958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1245.324804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_timeout -r 1 [ 1245.368863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e remote_timeout -r 1 [ 1245.415464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_timeout -r 1 [ 1245.452919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e remote_timeout -r 1 [ 1245.486420] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e remote_timeout -r 1 [ 1245.520718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e remote_timeout -r 1 [ 1245.559208] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e remote_timeout -r 1 [ 1245.599350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e remote_timeout -r 1 [ 1245.639261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.117@tcp [ 1245.663491] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1245.992817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1246.022594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1246.049502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1246.089599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1246.121076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1246.157970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1246.197543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1246.227827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1246.262835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1246.281422] LNet: 64736:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1246.299840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1246.331716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1246.359806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1246.402311] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1246.433085] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1246.462859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1246.493231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1246.535456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1251.463442] LNet: 64814:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1251.469858] LNetError: 64814:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1251.483218] LNet: Removed LNI 192.168.201.17@tcp [ 1251.507367] LNet: Removed LNI 192.168.201.17@tcp1 [ 1251.997849] Key type .llcrypt unregistered [ 1252.004074] Key type ._llcrypt unregistered [ 1269.515903] Key type ._llcrypt registered [ 1269.519313] Key type .llcrypt registered [ 1270.606250] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 00:30:56 (1789446656) [ 1272.247774] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_208 kernel ip2nets deprecated in 2.17.53 [ 1274.055258] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 00:30:59 (1789446659) [ 1274.750515] Key type .llcrypt unregistered [ 1274.752527] Key type ._llcrypt unregistered [ 1281.719283] Key type ._llcrypt registered [ 1281.722388] Key type .llcrypt registered [ 1281.899905] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1281.914746] alg: No test for adler32 (adler32-zlib) [ 1282.882349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1282.916340] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1290.864433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1291.509655] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e network_timeout -r 1 [ 1291.558448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e network_timeout -r 1 [ 1291.591992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1291.618649] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1291.870754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1291.903610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1291.939338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1291.973675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1292.002757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1292.036828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1292.078356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1292.115641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1292.130900] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1292.133956] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1292.160269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1292.179720] LNet: 66355:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1292.194608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1292.223540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1292.251719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1292.292951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1292.331950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1292.379953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1292.715556] LNet: 66394:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1292.726862] LNetError: 66394:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1292.741404] LNet: Removed LNI 192.168.201.17@tcp [ 1293.174823] Key type .llcrypt unregistered [ 1293.179684] Key type ._llcrypt unregistered [ 1315.838074] Key type ._llcrypt registered [ 1315.840076] Key type .llcrypt registered [ 1316.031954] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1316.054193] alg: No test for adler32 (adler32-zlib) [ 1317.043230] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1317.071642] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1324.976201] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1334.761914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1334.877778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1334.901622] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1336.255144] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e network_timeout -r 1 [ 1336.300496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e network_timeout -r 1 [ 1336.361542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e network_timeout -r 1 [ 1336.413562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e network_timeout -r 1 [ 1336.470909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e network_timeout -r 1 [ 1336.522505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e network_timeout -r 1 [ 1336.556523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e network_timeout -r 1 [ 1336.610214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e network_timeout -r 1 [ 1336.661371] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1336.691949] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1336.974759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1337.016594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1337.087883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1337.123333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1337.184831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1337.218502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1337.275745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1337.314776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1337.358967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1337.384497] LNet: 67273:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1337.423749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1337.456977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1337.512899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1337.538913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1337.567359] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1337.569138] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1337.583758] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.117@tcp [ 1337.595252] LNet: Skipped 1 previous similar message [ 1337.595733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1337.632716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1343.422188] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1343.976799] LNet: 67347:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1343.992513] LNetError: 67347:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1344.007261] LNet: Removed LNI 192.168.201.17@tcp [ 1344.024782] LNet: Removed LNI 192.168.201.17@tcp1 [ 1344.602127] Key type .llcrypt unregistered [ 1344.603517] Key type ._llcrypt unregistered [ 1363.273213] Key type ._llcrypt registered [ 1363.275019] Key type .llcrypt registered [ 1365.176919] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 00:32:29 (1789446749) [ 1365.739797] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1365.767342] alg: No test for adler32 (adler32-zlib) [ 1366.823755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1366.938075] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1367.017840] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1368.130982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1368.190821] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1369.315258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 1369.449178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 1369.494959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 1369.527369] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 1369.571437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 1369.621636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1369.652054] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.17@tcp [ 1369.677487] LNetError: 68447:0:(api-ni.c:4478:lnet_ni_set_healthv()) manually adding local NI 192.168.201.17@tcp to recovery [ 1369.951331] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1369.954179] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1372.000345] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1373.291974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1375.077146] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1375.895300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1375.941222] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1376.022752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1376.052785] LNet: 68544:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1376.068588] LNetError: 68544:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1376.077415] LNet: Removed LNI 192.168.201.17@tcp [ 1376.086267] LNet: Removed LNI 192.168.201.17@tcp1 [ 1376.118244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1376.209322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1376.245891] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1377.481955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1377.498423] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1378.559378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 1378.604938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1378.781095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 1378.905376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 1378.989415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 1379.100835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1379.156125] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1379.201941] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.17@tcp [ 1379.232397] LNetError: 68817:0:(api-ni.c:4478:lnet_ni_set_healthv()) manually adding local NI 192.168.201.17@tcp to recovery [ 1380.258653] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1382.765392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1386.400586] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1386.405639] LNet: Skipped 1 previous similar message [ 1391.360187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1391.414576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1391.453716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1399.451383] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 00:33:04 (1789446784) [ 1399.531121] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1399.554473] LNet: 69527:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1399.563960] LNetError: 69527:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1399.579984] LNetError: 69527:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1399.592242] LNet: Removed LNI 192.168.201.17@tcp [ 1399.608377] LNet: Removed LNI 192.168.201.17@tcp1 [ 1399.632037] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1399.684215] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1399.709648] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1400.284318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1400.305994] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1401.279322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 1401.339572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 1401.367220] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 1401.394383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 1401.460691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1401.536549] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1401.564210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.17@tcp --health 0 [ 1401.696100] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.17@tcp [ 1401.703795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1402.719436] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1402.726443] LNet: Skipped 1 previous similar message [ 1416.769920] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1416.935783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1417.002575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1417.048161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.17@tcp --health 500 [ 1417.217526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1424.244049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1424.381858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1424.419864] LNet: 70018:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1424.429797] LNetError: 70018:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1424.433228] LNetError: 70018:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1424.447406] LNet: Removed LNI 192.168.201.17@tcp [ 1424.473052] LNet: Removed LNI 192.168.201.17@tcp1 [ 1424.495366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1424.544841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1424.610652] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1425.596949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1425.614609] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1426.612640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 1426.655132] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1426.764565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 1426.798393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 1426.840772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 1426.886787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1426.929135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1426.971742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.17@tcp --health 0 [ 1427.618301] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1427.628830] LNet: Skipped 3 previous similar messages [ 1428.442947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1439.176889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1439.236761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1439.285103] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1448.672734] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 00:33:53 (1789446833) [ 1449.376733] LNet: 71065:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1449.389295] LNetError: 71065:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1449.403816] LNetError: 71065:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1449.425207] LNet: Removed LNI 192.168.201.17@tcp [ 1449.457720] LNet: Removed LNI 192.168.201.17@tcp1 [ 1450.432330] Key type .llcrypt unregistered [ 1450.440230] Key type ._llcrypt unregistered [ 1457.787596] Key type ._llcrypt registered [ 1457.789851] Key type .llcrypt registered [ 1458.051903] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1458.071643] alg: No test for adler32 (adler32-zlib) [ 1459.083961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1459.155484] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1471.374780] Lustre: DEBUG MARKER: Initial discovery [ 1471.544776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.117@tcp [ 1474.716349] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag [ 1475.201896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 -e local_error [ 1475.341845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.117@tcp [ 1475.371389] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1475.398867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1475.476616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d 192.168.201.117@tcp -r 1 -m GET -l 3 [ 1475.540124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 192.168.201.117@tcp -r 1 -m GET -e local_error [ 1475.551290] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1475.644835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d 192.168.201.117@tcp -r 1 -m PUT -l 6 [ 1478.053217] Lustre: DEBUG MARKER: Force oleg117-client.virtnet to discover 192.168.201.117@tcp (in background) [ 1478.360269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.117@tcp [ 1479.609821] Lustre: DEBUG MARKER: Force oleg117-server to discover 192.168.201.17@tcp [ 1481.448626] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1482.039163] Lustre: DEBUG MARKER: Wait for 71827 [ 1484.155337] Lustre: DEBUG MARKER: Finished wait on 71827 [ 1484.224331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay del -a [ 1484.272113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1484.272133] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1484.504672] LNet: 72278:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1484.510114] LNetError: 72278:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1484.526792] LNet: Removed LNI 192.168.201.17@tcp [ 1484.798704] Key type .llcrypt unregistered [ 1484.800931] Key type ._llcrypt unregistered [ 1503.844740] Key type ._llcrypt registered [ 1503.846496] Key type .llcrypt registered [ 1505.916819] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 00:34:50 (1789446890) [ 1506.690366] Key type .llcrypt unregistered [ 1506.692063] Key type ._llcrypt unregistered [ 1513.925083] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1514.231374] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1514.236562] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1514.246123] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1516.814741] Key type ._llcrypt registered [ 1516.817342] Key type .llcrypt registered [ 1517.099340] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1517.113663] alg: No test for adler32 (adler32-zlib) [ 1518.000122] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1518.069434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1518.102786] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1518.867545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1518.900227] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1526.858914] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 00:35:12 (1789446912) [ 1526.981629] LNet: 74345:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1526.993189] LNetError: 74345:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1526.999397] LNet: Removed LNI 10.1.2.3@tcp [ 1527.022164] LNet: Removed LNI 192.168.201.17@tcp [ 1527.676701] Key type .llcrypt unregistered [ 1527.681891] Key type ._llcrypt unregistered [ 1534.396901] Key type ._llcrypt registered [ 1534.399299] Key type .llcrypt registered [ 1534.542951] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1534.558749] alg: No test for adler32 (adler32-zlib) [ 1535.496286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1535.566408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1535.588540] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1536.315778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1536.334229] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1537.225763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 10.1.2.3@tcp 192.168.201.17@tcp [ 1538.336652] LNetError: Unexpected error -22 connecting to 192.168.201.17@tcp at host 192.168.201.17:988 [ 1543.470566] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 00:35:29 (1789446929) [ 1543.741752] LNet: 75482:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1543.749784] LNetError: 75482:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1543.754923] LNet: Removed LNI 10.1.2.3@tcp [ 1543.770099] LNet: Removed LNI 192.168.201.17@tcp [ 1544.333622] Key type .llcrypt unregistered [ 1544.336716] Key type ._llcrypt unregistered [ 1551.099763] Key type ._llcrypt registered [ 1551.103544] Key type .llcrypt registered [ 1551.402722] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1551.420995] alg: No test for adler32 (adler32-zlib) [ 1552.281540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1552.327478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1552.353102] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1552.945972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1552.960981] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 1553.791289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 192.168.201.17@tcp1 --nid 192.168.201.17@tcp2 [ 1558.027942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp1 192.168.201.17@tcp2 [ 1558.045419] LNetError: 76314:0:(lib-move.c:2094:lnet_handle_find_routed_path()) Unable to send message from 192.168.201.17@tcp1 to 192.168.201.17@tcp2 - Route table may be misconfigured [ 1558.059975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp2 192.168.201.17@tcp1 [ 1563.759751] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 00:35:49 (1789446949) [ 1563.851357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1563.876748] LNet: 76881:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1563.882345] LNetError: 76881:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1563.889295] LNet: Removed LNI 192.168.201.17@tcp1 [ 1564.905176] LNet: Removed LNI 192.168.201.17@tcp2 [ 1564.948661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1565.036308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1565.088111] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1565.933683] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1565.966726] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1566.762556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 1566.837363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.17@tcp [ 1566.895330] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp1 192.168.201.17@tcp1 [ 1566.943750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -r 1 -s 192.168.201.17@tcp -d 192.168.201.17@tcp -e network_timeout [ 1566.978118] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -e network_timeout [ 1567.046807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -e network_timeout [ 1567.108092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -e network_timeout [ 1567.171113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1567.193896] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1567.209150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1567.300447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1567.411183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1568.098929] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1574.899726] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 00:36:00 (1789446960) [ 1574.985786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1575.012085] LNet: 77741:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1575.020441] LNetError: 77741:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1575.023578] LNetError: 77741:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1575.037467] LNet: Removed LNI 192.168.201.17@tcp [ 1576.105521] LNet: Removed LNI 192.168.201.17@tcp1 [ 1576.155057] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1576.340213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 0@lo [ 1576.596277] LNet: 77786:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1577.120726] Key type .llcrypt unregistered [ 1577.128135] Key type ._llcrypt unregistered [ 1588.499691] Key type ._llcrypt registered [ 1588.504349] Key type .llcrypt registered [ 1589.579087] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 00:36:15 (1789446975) [ 1589.944710] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1589.952835] alg: No test for adler32 (adler32-zlib) [ 1590.939189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1592.616358] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces [ 1594.233637] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 00:36:19 (1789446979) [ 1594.494575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1594.517465] LNet: 78881:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1594.535284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1594.638645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1594.679212] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1595.358111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1595.415852] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1596.310373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1596.366599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp1 [ 1596.425071] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp1 [ 1602.817499] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 00:36:28 (1789446988) [ 1605.005558] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_220 Need at least 2 remote nodes found oleg117-server [ 1606.876153] Lustre: DEBUG MARKER: == sanity-lnet test 221: Add routes w/DD disabled - check aliveness ========================================================== 00:36:32 (1789446992) [ 1608.521879] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_221 Need at least 2 remote nodes found oleg117-server [ 1610.440496] Lustre: DEBUG MARKER: == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 00:36:36 (1789446996) [ 1612.716823] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_222 Need at least 2 remote nodes found oleg117-server [ 1614.276471] Lustre: DEBUG MARKER: == sanity-lnet test 223: Check avoid_asym_router_failure=1 w/DD disabled ========================================================== 00:36:39 (1789446999) [ 1616.617698] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_223 Need at least 2 remote nodes found oleg117-server [ 1618.909904] Lustre: DEBUG MARKER: == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 00:36:43 (1789447003) [ 1621.350476] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_224 Need at least 2 remote nodes found oleg117-server [ 1623.401451] Lustre: DEBUG MARKER: == sanity-lnet test 225: Check avoid_asym_router_failure=0 w/DD disabled ========================================================== 00:36:48 (1789447008) [ 1625.229855] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_225 Need at least 2 remote nodes found oleg117-server [ 1626.960174] Lustre: DEBUG MARKER: == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 00:36:52 (1789447012) [ 1629.607826] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_226 Need at least 3 remote nodes found oleg117-server [ 1631.658150] Lustre: DEBUG MARKER: == sanity-lnet test 227: Check router peer health w/DD disabled ========================================================== 00:36:56 (1789447016) [ 1634.653814] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_227 Need at least 3 remote nodes found oleg117-server [ 1637.627443] Lustre: DEBUG MARKER: == sanity-lnet test 228: Routes should stay up when health is decremented ========================================================== 00:37:02 (1789447022) [ 1639.947465] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_228 Need at least 3 remote nodes found oleg117-server [ 1641.745681] Lustre: DEBUG MARKER: == sanity-lnet test 230: Test setting conns-per-peer ===== 00:37:07 (1789447027) [ 1642.016657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1642.061408] LNet: 82757:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1642.078632] LNetError: 82757:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1642.092593] LNet: Removed LNI 192.168.201.17@tcp [ 1642.135664] LNet: Removed LNI 192.168.201.17@tcp1 [ 1642.202630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1642.309056] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1642.347388] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1643.101923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 4 [ 1643.161938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1643.311410] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1643.331604] LNet: 82906:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1643.337615] LNetError: 82906:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1643.343270] LNetError: 82906:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1643.356280] LNet: Removed LNI 192.168.201.17@tcp [ 1643.379299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1643.430606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 5 [ 1643.456624] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1644.299788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1644.418515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1644.434260] LNetError: 83051:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1644.447715] LNet: Removed LNI 192.168.201.17@tcp [ 1644.460043] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1644.499078] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1644.518409] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1645.018316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 6 [ 1645.089969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1645.218277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1646.256083] LNet: Removed LNI 192.168.201.17@tcp [ 1646.283661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1646.346637] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 7 [ 1646.365131] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1646.979848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1647.234959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1647.268548] LNet: 83343:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1647.277685] LNet: 83343:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 1647.292644] LNetError: 83343:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1647.303953] LNetError: 83343:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1647.325257] LNet: Removed LNI 192.168.201.17@tcp [ 1647.374867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1647.489883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1647.536300] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1648.246859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 8 [ 1648.325366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1648.588199] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1649.638973] LNet: Removed LNI 192.168.201.17@tcp [ 1649.694459] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1649.733289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 9 [ 1649.750307] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1650.466496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1650.856655] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1651.954886] LNet: Removed LNI 192.168.201.17@tcp [ 1651.991774] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1652.055550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1652.086489] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1653.066117] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 10 [ 1653.186954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1654.464486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1654.500319] LNet: 83793:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1654.508164] LNet: 83793:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 1654.513949] LNetError: 83793:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1654.526087] LNetError: 83793:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 1654.542884] LNet: Removed LNI 192.168.201.17@tcp [ 1654.579712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1654.642971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 11 [ 1654.694149] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1655.759794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1656.050446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1657.125421] LNet: Removed LNI 192.168.201.17@tcp [ 1657.241531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1657.395951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1657.479967] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1658.486483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 12 [ 1658.624603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1658.978671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1660.007848] LNet: Removed LNI 192.168.201.17@tcp [ 1660.047362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1660.195944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 13 [ 1660.247482] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1661.645444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1662.038139] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1663.078973] LNet: Removed LNI 192.168.201.17@tcp [ 1663.160637] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1663.259464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1663.298124] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1664.083129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 14 [ 1664.192190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1664.513669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1664.594097] LNet: 84378:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1664.602587] LNet: 84378:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 1664.619610] LNetError: 84378:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1664.626832] LNetError: 84378:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 1664.642973] LNet: Removed LNI 192.168.201.17@tcp [ 1664.682335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1664.768970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 15 [ 1664.789541] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1665.903929] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1667.301552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1668.391329] LNet: Removed LNI 192.168.201.17@tcp [ 1668.432653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1668.522827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1668.556705] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1669.455928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 128 [ 1669.531630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1669.560419] LNet: Removed LNI 192.168.201.17@tcp [ 1669.585452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1669.633261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1669.678461] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1670.607329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer -1 [ 1670.660658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1670.704620] LNet: Removed LNI 192.168.201.17@tcp [ 1670.740222] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1670.823273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 128 [ 1670.907496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1670.968739] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1671.044730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 1000000000000000000000000 [ 1671.141593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1671.186653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1671.238846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer foo [ 1671.343867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1671.394734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1671.454362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 8 [ 1671.482924] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1672.567638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1672.607633] LNet: Removed LNI 192.168.201.17@tcp [ 1672.631135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1672.704813] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 1679.531684] Lustre: DEBUG MARKER: == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 00:37:45 (1789447065) [ 1679.636661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1679.705428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1679.747989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 [ 1679.768049] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/180] [ 1681.462322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1681.491128] LNet: 85692:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1681.497219] LNet: 85692:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 8 previous similar messages [ 1681.504206] LNetError: 85692:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1681.511700] LNetError: 85692:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 4 previous similar messages [ 1681.542703] LNet: Removed LNI 192.168.201.17@tcp231 [ 1681.562341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1681.695840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1681.736785] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/180] [ 1681.750123] LNet: 85700:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1682.350790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1682.398604] LNet: Removed LNI 192.168.201.17@tcp231 [ 1682.420076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=-1 [ 1682.449967] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/180] [ 1683.986761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1684.027633] LNet: Removed LNI 192.168.201.17@tcp231 [ 1684.059755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1684.120333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1684.143308] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/0] [ 1684.149469] LNet: 85852:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1684.666276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1684.702471] LNet: Removed LNI 192.168.201.17@tcp231 [ 1684.732890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=0 [ 1684.770457] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/0] [ 1686.132354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1686.191253] LNet: Removed LNI 192.168.201.17@tcp231 [ 1686.233174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1686.301712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1686.348932] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/60] [ 1686.353587] LNet: 86004:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1686.887735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1686.941167] LNet: Removed LNI 192.168.201.17@tcp231 [ 1686.962747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=60 [ 1686.985299] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/60] [ 1688.102555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1688.135937] LNet: Removed LNI 192.168.201.17@tcp231 [ 1688.151066] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1688.291668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1688.322253] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/180] [ 1688.675984] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1688.706787] LNet: Removed LNI 192.168.201.17@tcp231 [ 1688.735120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=180 [ 1688.765670] LNet: Added LNI 192.168.201.17@tcp231 [8/256/0/180] [ 1696.821638] Lustre: DEBUG MARKER: == sanity-lnet test 232: Test setting ToS value ========== 00:38:02 (1789447082) [ 1697.191837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1697.224635] LNet: Removed LNI 192.168.201.17@tcp231 [ 1697.241128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1697.317367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 [ 1697.346043] LNet: Added LNI 192.168.201.17@tcp232 [8/256/0/180] [ 1698.179168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1698.249843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1698.291679] LNet: Removed LNI 192.168.201.17@tcp232 [ 1698.309309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos -1 [ 1698.347148] LNet: Added LNI 192.168.201.17@tcp232 [8/256/0/180] [ 1699.222843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1699.262876] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1699.302096] LNet: Removed LNI 192.168.201.17@tcp232 [ 1699.331927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 104 [ 1699.368082] LNet: Added LNI 192.168.201.17@tcp232 [8/256/0/180] [ 1700.265662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1700.326178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1700.372085] LNet: Removed LNI 192.168.201.17@tcp232 [ 1700.390239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 106 [ 1700.449388] LNet: Added LNI 192.168.201.17@tcp232 [8/256/0/180] [ 1701.189811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1701.228303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1701.264768] LNet: Removed LNI 192.168.201.17@tcp232 [ 1701.280067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 300 [ 1707.819185] Lustre: DEBUG MARKER: == sanity-lnet test 233: Check for successful resends ==== 00:38:13 (1789447093) [ 1708.876744] Key type .llcrypt unregistered [ 1708.879185] Key type ._llcrypt unregistered [ 1715.634649] Key type ._llcrypt registered [ 1715.636570] Key type .llcrypt registered [ 1715.889216] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1715.915401] alg: No test for adler32 (adler32-zlib) [ 1716.954191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1716.980502] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1724.638116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1734.493639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1734.576741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1734.613636] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1736.034760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_interrupt -r 2 [ 1736.063954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_interrupt -r 2 [ 1736.090814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_interrupt -r 2 [ 1736.120864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_interrupt -r 2 [ 1736.172352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_interrupt -r 2 [ 1736.205938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_interrupt -r 2 [ 1736.244670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_interrupt -r 2 [ 1736.263665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_interrupt -r 2 [ 1736.291053] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1736.308309] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1736.311173] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1736.574949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1736.605900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1736.632352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1736.661786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1736.690657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1736.722566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1736.761903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1736.786375] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1736.827341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1736.842506] LNet: 88656:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1736.874502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1736.909488] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1736.935925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1736.965367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1737.008921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1737.050233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1737.674500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_dropped -r 2 [ 1737.715595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_dropped -r 2 [ 1737.751773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_dropped -r 2 [ 1737.792068] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_dropped -r 2 [ 1737.831794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_dropped -r 2 [ 1737.871576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_dropped -r 2 [ 1737.915959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_dropped -r 2 [ 1737.971790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_dropped -r 2 [ 1738.023586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1738.047652] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1738.376635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1738.448677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1738.502633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1738.528324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1738.551530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1738.578189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1738.599765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1738.628557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1738.654660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1738.669149] LNet: 88778:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1738.695396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1738.720128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1738.763400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1738.800231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1738.827891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1738.858704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1739.455475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_aborted -r 2 [ 1739.507461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_aborted -r 2 [ 1739.583082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_aborted -r 2 [ 1739.654928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_aborted -r 2 [ 1739.714230] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_aborted -r 2 [ 1739.754570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_aborted -r 2 [ 1739.814579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_aborted -r 2 [ 1739.877994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_aborted -r 2 [ 1739.921424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1739.943537] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1740.326244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1740.395573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1740.440523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1740.470914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1740.502533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1740.532207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1740.563596] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1740.603130] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1740.653044] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1740.679538] LNet: 88900:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1740.699636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1740.743311] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1740.773681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1740.802870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1740.837523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1740.886698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1741.456857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_no_route -r 2 [ 1741.496868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_no_route -r 2 [ 1741.550781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_no_route -r 2 [ 1741.585585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_no_route -r 2 [ 1741.619469] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_no_route -r 2 [ 1741.656704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_no_route -r 2 [ 1741.689493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_no_route -r 2 [ 1741.739687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_no_route -r 2 [ 1741.783963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1742.018517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1742.051944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1742.118714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1742.182672] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1742.253148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1742.301927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1742.350720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1742.400675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1742.452727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1742.499678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1742.528498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1742.559834] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1742.592563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1742.632149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1742.666685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1743.222315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e local_timeout -r 2 [ 1743.259571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e local_timeout -r 2 [ 1743.292840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e local_timeout -r 2 [ 1743.321199] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e local_timeout -r 2 [ 1743.364978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e local_timeout -r 2 [ 1743.390815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e local_timeout -r 2 [ 1743.423813] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e local_timeout -r 2 [ 1743.457437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e local_timeout -r 2 [ 1743.482921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1743.501531] LNet: There was an unexpected network error while writing to 192.168.201.117: rc = -22 [ 1743.509667] LNet: Skipped 1 previous similar message [ 1743.814682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1743.848843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1743.871629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1743.907833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1743.941416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1743.974588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1744.025295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1744.056950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1744.079169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1744.102565] LNet: 89145:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1744.109660] LNet: 89145:0:(api-ni.c:301:recovery_interval_set()) Skipped 1 previous similar message [ 1744.132301] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1744.183290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1744.225603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1744.271902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1744.316501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1744.368291] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1744.957797] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp -m GET -e remote_dropped -r 2 [ 1744.993843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.117@tcp1 -m GET -e remote_dropped -r 2 [ 1745.035935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp -m GET -e remote_dropped -r 2 [ 1745.088947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp -d 192.168.201.17@tcp1 -m GET -e remote_dropped -r 2 [ 1745.141099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp -m GET -e remote_dropped -r 2 [ 1745.189495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.117@tcp1 -m GET -e remote_dropped -r 2 [ 1745.231500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp -m GET -e remote_dropped -r 2 [ 1745.262093] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.17@tcp1 -d 192.168.201.17@tcp1 -m GET -e remote_dropped -r 2 [ 1745.295350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.17@tcp 192.168.201.117@tcp [ 1745.314420] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.117@tcp [ 1745.559579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1745.601599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1745.630944] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1745.673195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1745.716090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 1745.754600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1745.793618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1745.835861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1745.895473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1745.931145] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1745.987647] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1746.040447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1746.073465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1746.124828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1746.191390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1752.116302] LNet: 89342:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1752.126509] LNetError: 89342:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1752.136081] LNet: Removed LNI 192.168.201.17@tcp [ 1752.158647] LNet: Removed LNI 192.168.201.17@tcp1 [ 1752.650256] Key type .llcrypt unregistered [ 1752.653845] Key type ._llcrypt unregistered [ 1768.937317] Key type ._llcrypt registered [ 1768.940963] Key type .llcrypt registered [ 1770.081679] Lustre: DEBUG MARKER: == sanity-lnet test 234: Verify NI status changes with manual health value transitions ========================================================== 00:39:15 (1789447155) [ 1770.483969] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1770.508368] alg: No test for adler32 (adler32-zlib) [ 1771.462583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1771.522668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1771.561806] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1772.526138] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 0 [ 1772.557082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.17@tcp [ 1772.599440] LNetError: 90318:0:(api-ni.c:4478:lnet_ni_set_healthv()) manually adding local NI 192.168.201.17@tcp to recovery [ 1772.746963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --nid 192.168.201.17@tcp [ 1772.931076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1781.334421] Lustre: DEBUG MARKER: == sanity-lnet test 235: Verify NI status DOWN when health reaches 0 after failures ========================================================== 00:39:25 (1789447165) [ 1781.535606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1781.591784] LNet: 90907:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1781.610233] LNetError: 90907:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1781.648203] LNet: Removed LNI 192.168.201.17@tcp [ 1781.671895] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1781.720237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1781.748332] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1782.308926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1782.326938] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 1783.491790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 100 --nid 192.168.201.17@tcp [ 1783.520276] LNetError: 91176:0:(api-ni.c:4478:lnet_ni_set_healthv()) manually adding local NI 192.168.201.17@tcp to recovery [ 1783.775550] LNet: 1 local NIs in recovery (showing 1): 192.168.201.17@tcp [ 1783.792672] LNet: There was an unexpected network error while writing to 192.168.201.17: rc = -22 [ 1784.851361] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 900 --nid 192.168.201.17@tcp [ 1792.797698] Lustre: DEBUG MARKER: == sanity-lnet test 236: Local NI state propagates to routers ========================================================== 00:39:38 (1789447178) [ 1794.859995] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_236 Need at least 2 remote nodes found oleg117-server [ 1797.150904] Lustre: DEBUG MARKER: == sanity-lnet test 237: Check MR forwarding feature ===== 00:39:42 (1789447182) [ 1799.022642] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_237 Need at least 2 remote nodes found oleg117-server [ 1801.144971] Lustre: DEBUG MARKER: == sanity-lnet test 238: Check MR forwarding feature skipped for non-MR source ========================================================== 00:39:46 (1789447186) [ 1803.122417] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_238 Need at least 2 remote nodes found oleg117-server [ 1804.996739] Lustre: DEBUG MARKER: == sanity-lnet test 241: Check setting LND timeout value via lnetctl updates tunables ========================================================== 00:39:50 (1789447190) [ 1805.116563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1805.161744] LNet: 92793:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1805.174969] LNetError: 92793:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1805.187127] LNet: Removed LNI 192.168.201.17@tcp [ 1805.212270] LNet: Removed LNI 192.168.201.17@tcp1 [ 1805.251587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1805.324106] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1805.342953] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1805.910103] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set -h [ 1805.961607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 51 [ 1805.993608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp -v [ 1806.072913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 0 [ 1806.110708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp -v [ 1806.176260] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 50 [ 1813.022150] Lustre: DEBUG MARKER: == sanity-lnet test 245: Check read-only params do not appear in export --backup ========================================================== 00:39:58 (1789447198) [ 1813.282222] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1813.321649] LNet: 93531:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1813.332177] LNetError: 93531:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1813.338271] LNetError: 93531:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1813.374298] LNet: Removed LNI 192.168.201.17@tcp [ 1813.400649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1813.462583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1813.486063] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1820.764752] Lustre: DEBUG MARKER: == sanity-lnet test 250: test that linux routes are added ========================================================== 00:40:06 (1789447206) [ 1821.397602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1821.421819] LNet: 94236:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1821.424778] LNetError: 94236:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1821.437993] LNet: Removed LNI 192.168.201.17@tcp [ 1821.451187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1821.509938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1821.559864] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1828.984624] Lustre: DEBUG MARKER: == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 00:40:14 (1789447214) [ 1831.044042] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_251 Need kfi NETTYPE [ 1833.150434] Lustre: DEBUG MARKER: == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 00:40:18 (1789447218) [ 1834.004268] LNet: 95271:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1834.012557] LNetError: 95271:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1834.028833] LNet: Removed LNI 192.168.201.17@tcp [ 1834.770724] Key type .llcrypt unregistered [ 1834.772250] Key type ._llcrypt unregistered [ 1843.609512] Key type ._llcrypt registered [ 1843.618131] Key type .llcrypt registered [ 1844.042270] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1844.074671] alg: No test for adler32 (adler32-zlib) [ 1845.079120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1845.135432] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1854.373591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 1862.329960] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --timeout 15 192.168.201.117@tcp [ 1862.890194] LNet: 95687:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1862.897528] LNetError: 95687:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1862.909916] LNet: Removed LNI 192.168.201.17@tcp [ 1863.357628] Key type .llcrypt unregistered [ 1863.360442] Key type ._llcrypt unregistered [ 1877.016346] Key type ._llcrypt registered [ 1877.024839] Key type .llcrypt registered [ 1878.269605] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253 [ 1880.120461] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254 [ 1882.356823] Lustre: DEBUG MARKER: == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 00:41:07 (1789447267) [ 1884.303497] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_255 kernel ip2nets deprecated in 2.17.53 [ 1886.598801] Lustre: DEBUG MARKER: == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 00:41:11 (1789447271) [ 1888.154202] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_256 Need at least 2 remote nodes found oleg117-server [ 1889.936421] Lustre: DEBUG MARKER: == sanity-lnet test 257: Test graceful router shutdown/startup ========================================================== 00:41:15 (1789447275) [ 1891.880487] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_257 Need at least 3 remote nodes found oleg117-server [ 1894.287379] Lustre: DEBUG MARKER: == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 00:41:19 (1789447279) [ 1896.568556] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1896.621089] alg: No test for adler32 (adler32-zlib) [ 1898.042941] Lustre: Lustre: Build Version: 2.17.58_39_ge07ba0b [ 1898.533823] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1900.407357] Key type lgssc registered [ 1902.709594] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1926.291596] Key type lgssc unregistered [ 1926.548467] LNet: 98862:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1926.554821] LNetError: 98862:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1926.566895] LNet: Removed LNI 192.168.201.17@tcp [ 1927.384705] Key type .llcrypt unregistered [ 1927.388361] Key type ._llcrypt unregistered [ 1943.358713] Key type ._llcrypt registered [ 1943.360676] Key type .llcrypt registered [ 1944.883291] Lustre: DEBUG MARKER: == sanity-lnet test 265: Import of invalid CPT should fail ========================================================== 00:42:10 (1789447330) [ 1945.306473] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1945.341522] alg: No test for adler32 (adler32-zlib) [ 1946.265953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1946.364526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 1946.417604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 1946.464567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 1946.529154] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 1952.270603] Lustre: DEBUG MARKER: == sanity-lnet test 266: Validate CPT parsing in network sequence ========================================================== 00:42:18 (1789447338) [ 1952.489793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1952.531361] LNet: 100274:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1952.607896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1952.703043] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1952.737782] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1953.621785] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1953.632153] LNet: 100418:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1953.635525] LNetError: 100418:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1953.651545] LNet: Removed LNI 192.168.201.17@tcp [ 1953.662826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1953.708515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 1953.742505] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1954.456216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1954.513611] LNetError: 100446:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1954.531946] LNet: Removed LNI 192.168.201.17@tcp [ 1954.577494] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1954.619377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 1954.637333] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1962.276987] Lustre: DEBUG MARKER: == sanity-lnet test 270: Test NID metadata =============== 00:42:27 (1789447347) [ 1962.542542] LNet: 101035:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1962.559158] LNet: 101035:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 1962.569698] LNetError: 101035:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1962.621363] LNet: Removed LNI 192.168.201.17@tcp [ 1963.372749] Key type .llcrypt unregistered [ 1963.376181] Key type ._llcrypt unregistered [ 1970.450657] Key type ._llcrypt registered [ 1970.453375] Key type .llcrypt registered [ 1970.659907] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1970.676369] alg: No test for adler32 (adler32-zlib) [ 1971.661442] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1971.766804] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 1971.774977] LNet: *** cfs_fail_loc=e003, val=100*** [ 1972.485714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 1972.507730] LNet: *** cfs_fail_loc=e003, val=100*** [ 1972.510583] LNet: Skipped 2 previous similar messages [ 1972.517512] Ping reply received with mapping: [ 1972.521177] found metadata reply NI[192.168.201.17@tcp] [ 1972.524615] 11 c9 a8 c0 00 00 02 00 5a 5a 5a 5a 5a 5a 5a 5a [ 1972.528550] 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a [ 1972.534981] 5a 5a 5a 5a 5a 5a 5a 5a [ 1972.746995] LNet: 101431:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1972.752226] LNetError: 101431:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1973.805628] LNet: Removed LNI 192.168.201.17@tcp [ 1974.626600] Key type .llcrypt unregistered [ 1974.630499] Key type ._llcrypt unregistered [ 1987.594895] Key type ._llcrypt registered [ 1987.597180] Key type .llcrypt registered [ 1988.908213] Lustre: DEBUG MARKER: == sanity-lnet test 280: Don't panic when request_module fails ========================================================== 00:42:54 (1789447374) [ 1989.762855] Key type .llcrypt unregistered [ 1989.775205] Key type ._llcrypt unregistered [ 1995.970400] Key type ._llcrypt registered [ 1995.973738] Key type .llcrypt registered [ 1996.168364] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1996.193167] alg: No test for adler32 (adler32-zlib) [ 1997.283378] LNetError: 102469:0:(api-ni.c:2855:lnet_load_lnd()) Can't load LND tcp, module ksocklnd, rc=256 [ 1997.637511] Key type .llcrypt unregistered [ 1997.643123] Key type ._llcrypt unregistered [ 2003.095404] Key type ._llcrypt registered [ 2003.097614] Key type .llcrypt registered [ 2004.764128] Lustre: DEBUG MARKER: == sanity-lnet test 290: Check that lnet acceptor is using explicit address binding ========================================================== 00:43:09 (1789447389) [ 2005.400961] Key type .llcrypt unregistered [ 2005.402540] Key type ._llcrypt unregistered [ 2011.841941] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2011.856326] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2014.575994] Key type ._llcrypt registered [ 2014.578223] Key type .llcrypt registered [ 2014.757642] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2014.773434] alg: No test for adler32 (adler32-zlib) [ 2015.780632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2015.884291] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2026.223529] Lustre: DEBUG MARKER: == sanity-lnet test 291: load lnet using custom accept port for bulk tcp connections ========================================================== 00:43:31 (1789447411) [ 2026.367643] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2026.406822] LNet: 104239:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2026.422166] LNetError: 104239:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2026.446080] LNet: Removed LNI 10.1.2.3@tcp [ 2026.474573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2027.155875] Key type .llcrypt unregistered [ 2027.162039] Key type ._llcrypt unregistered [ 2034.341869] Key type ._llcrypt registered [ 2034.343864] Key type .llcrypt registered [ 2034.551186] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2034.566522] alg: No test for adler32 (adler32-zlib) [ 2035.498085] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2035.572938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2035.743691] LNet: 104529:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2035.751822] LNetError: 104529:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2036.776493] LNet: Removed LNI 192.168.201.17@tcp [ 2037.675554] Key type .llcrypt unregistered [ 2037.679402] Key type ._llcrypt unregistered [ 2047.987473] Key type ._llcrypt registered [ 2047.990222] Key type .llcrypt registered [ 2049.129656] Lustre: DEBUG MARKER: == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 00:43:54 (1789447434) [ 2049.893303] Key type .llcrypt unregistered [ 2049.898441] Key type ._llcrypt unregistered [ 2056.760210] Key type ._llcrypt registered [ 2056.766739] Key type .llcrypt registered [ 2056.997725] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2057.018820] alg: No test for adler32 (adler32-zlib) [ 2064.875491] Lustre: DEBUG MARKER: == sanity-lnet test 301: Check for dynamic adds of same/wrong interface (memory leak) ========================================================== 00:44:09 (1789447449) [ 2065.025697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2065.120802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2065.278501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2065.356741] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2066.296587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2066.351699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if ens2 [ 2066.415123] LNet: Removed LNI 192.168.201.17@tcp [ 2066.757433] LNet: 106268:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2067.266605] Key type .llcrypt unregistered [ 2067.271641] Key type ._llcrypt unregistered [ 2082.577205] Key type ._llcrypt registered [ 2082.579924] Key type .llcrypt registered [ 2083.985830] Lustre: DEBUG MARKER: == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 00:44:29 (1789447469) [ 2084.505052] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2084.536823] alg: No test for adler32 (adler32-zlib) [ 2085.635497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2085.714319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2085.742286] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2086.535862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2086.564562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug peer --nid 192.168.201.17@tcp [ 2086.579713] LNet: Conn 0 [type, closing, crefcnt, srefcnt]: 2, 0, 1, 1 [ 2086.581865] LNet: Conn 0 rx [scheduled, ready, state]: 0, 0, 1 [ 2086.584300] LNet: Conn 0 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2086, 0, 0 [ 2086.587802] LNet: Conn 0 sched [nconns, cpt]: 4, 0 [ 2086.589955] LNet: Conn 1 [type, closing, crefcnt, srefcnt]: 3, 0, 1, 1 [ 2086.592879] LNet: Conn 1 rx [scheduled, ready, state]: 0, 0, 1 [ 2086.594899] LNet: Conn 1 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2086, 0, 0 [ 2086.598484] LNet: Conn 1 sched [nconns, cpt]: 4, 0 [ 2086.600040] LNet: Conn 2 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 2086.606583] LNet: Conn 2 rx [scheduled, ready, state]: 0, 0, 1 [ 2086.611039] LNet: Conn 2 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2086, 0, 2136 [ 2086.615382] LNet: Conn 2 sched [nconns, cpt]: 4, 0 [ 2086.617441] LNet: Conn 3 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 2086.619968] LNet: Conn 3 rx [scheduled, ready, state]: 0, 0, 1 [ 2086.623691] LNet: Conn 3 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2086, 0, 2136 [ 2086.630394] LNet: Conn 3 sched [nconns, cpt]: 4, 0 [ 2094.724409] Lustre: DEBUG MARKER: == sanity-lnet test 303: Check peer NI health after link down ========================================================== 00:44:39 (1789447479) [ 2095.165333] LNet: 107808:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2095.173863] LNetError: 107808:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2096.230179] LNet: Removed LNI 192.168.201.17@tcp [ 2097.391739] Key type .llcrypt unregistered [ 2097.394456] Key type ._llcrypt unregistered [ 2103.984603] Key type ._llcrypt registered [ 2103.993126] Key type .llcrypt registered [ 2104.314820] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2104.346170] alg: No test for adler32 (adler32-zlib) [ 2105.473450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2105.543562] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2114.472690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 2124.255175] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2124.358189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2124.393959] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 2125.554646] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 2125.774048] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2125.783611] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2125.791039] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 2128.155743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp99 --if test1pg [ 2128.182376] LNet: Added LNI 10.1.2.3@tcp99 [8/256/0/180] [ 2138.615518] Lustre: DEBUG MARKER: == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 00:45:24 (1789447524) [ 2138.816958] LNet: 109313:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2138.824599] LNetError: 109313:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2139.871168] LNet: Removed LNI 192.168.201.17@tcp [ 2139.874951] LNetError: 109313:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2139.890471] LNet: Removed LNI 192.168.201.17@tcp1 [ 2139.910378] LNet: Removed LNI 10.1.2.3@tcp99 [ 2140.449620] Key type .llcrypt unregistered [ 2140.454066] Key type ._llcrypt unregistered [ 2146.826916] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2146.835424] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2149.600853] Key type ._llcrypt registered [ 2149.604183] Key type .llcrypt registered [ 2149.943871] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2149.977746] alg: No test for adler32 (adler32-zlib) [ 2150.992719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2151.098967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2151.166526] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2151.939196] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2151.957778] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2154.042654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.1.2.3@tcp --lock_prim [ 2154.093160] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2154.137702] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 10.1.2.3@tcp --state 1048577 [ 2154.155981] LNetError: 110056:0:(peer.c:2366:lnet_del_peer_ni()) peer 10.1.2.3@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 2154.189638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2154.219822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2154.533334] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2154.634473] LNet: 110076:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2154.643514] LNetError: 110076:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2154.654863] LNet: Removed LNI 10.1.2.3@tcp [ 2155.692680] LNet: Removed LNI 192.168.201.17@tcp [ 2156.355489] Key type .llcrypt unregistered [ 2156.358196] Key type ._llcrypt unregistered [ 2169.104625] Key type ._llcrypt registered [ 2169.115274] Key type .llcrypt registered [ 2170.290537] Lustre: DEBUG MARKER: == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 00:45:55 (1789447555) [ 2170.611086] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2170.633535] alg: No test for adler32 (adler32-zlib) [ 2171.630900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2171.692646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2171.725736] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2172.506176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2172.566344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping oleg117-client@tcp [ 2178.069519] Lustre: DEBUG MARKER: == sanity-lnet test 310: Set timeout and verify ========== 00:46:03 (1789447563) [ 2178.173404] LNet: 111600:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2178.191278] LNetError: 111600:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2178.219075] LNet: Removed LNI 192.168.201.17@tcp [ 2178.785408] Key type .llcrypt unregistered [ 2178.788388] Key type ._llcrypt unregistered [ 2184.842818] Key type ._llcrypt registered [ 2184.844957] Key type .llcrypt registered [ 2184.978370] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2184.989884] alg: No test for adler32 (adler32-zlib) [ 2185.885259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure --all [ 2185.909394] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2185.938628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp -v [ 2192.222922] Lustre: DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 00:46:17 (1789447577) [ 2193.520374] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type [ 2195.542305] Lustre: DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 00:46:21 (1789447581) [ 2197.328422] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type [ 2198.952547] Lustre: DEBUG MARKER: == sanity-lnet test 350: Check import --exec of ping/discover ========================================================== 00:46:24 (1789447584) [ 2199.143188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2199.189565] LNet: 113026:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2199.197742] LNetError: 113026:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2199.231050] LNet: Removed LNI 192.168.201.17@tcp [ 2199.273212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2199.359714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2199.392396] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2200.022900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 2200.054949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 2207.543962] Lustre: DEBUG MARKER: == sanity-lnet test 400: Check for udsp add/delete net rule without net num ========================================================== 00:46:32 (1789447592) [ 2207.647538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2207.686358] LNet: 113732:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2207.698727] LNetError: 113732:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2207.734250] LNet: Removed LNI 192.168.201.17@tcp [ 2207.780937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2207.879796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --priority 0 [ 2207.944200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2208.697177] Key type .llcrypt unregistered [ 2208.698532] Key type ._llcrypt unregistered [ 2222.053922] Key type ._llcrypt registered [ 2222.059563] Key type .llcrypt registered [ 2223.363017] Lustre: DEBUG MARKER: == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 00:46:48 (1789447608) [ 2223.793232] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2223.817126] alg: No test for adler32 (adler32-zlib) [ 2224.786371] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2224.849636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2224.888443] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2225.633247] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2225.684224] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2232.883721] Lustre: DEBUG MARKER: == sanity-lnet test 402: Destination net rule should not panic ========================================================== 00:46:58 (1789447618) [ 2233.002768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2233.052632] LNet: 115290:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2233.076400] LNetError: 115290:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2234.155572] LNet: Removed LNI 192.168.201.17@tcp [ 2234.205868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2234.289396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst kfi --priority 0 [ 2234.334500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 402@kfi [ 2240.808896] Lustre: DEBUG MARKER: == sanity-lnet test 403: Add and delete udsp net and nid rules ========================================================== 00:47:06 (1789447626) [ 2240.912820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2240.934905] LNet: 115867:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2240.990351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2241.079815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2241.573873] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2241.602272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2241.652393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src 1.1.1.1@tcp --prio 1 [ 2241.988368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2242.023694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2242.105680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2242.130350] LNet: 115905:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2242.156153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2242.254771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2242.778500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2242.809659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2242.851977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst 1.1.1.1@tcp --prio 1 [ 2243.234447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2243.264193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2248.738790] Lustre: DEBUG MARKER: == sanity-lnet test 404: Check udsp src/dst net rule ===== 00:47:14 (1789447634) [ 2248.904578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2248.937414] LNet: 116502:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2248.964612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2249.068799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2249.113327] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2249.865475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2249.896437] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2250.804446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2250.876742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2251.094210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2251.123907] LNetError: 116785:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2251.135637] LNet: Removed LNI 192.168.201.17@tcp [ 2251.166506] LNet: Removed LNI 192.168.201.17@tcp2 [ 2251.214724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2251.312431] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2251.316928] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2251.321524] LNet: 116791:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2251.868943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2252.074162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2253.154114] LNet: Removed LNI 192.168.201.17@tcp [ 2253.155948] LNetError: 116828:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2253.161465] LNetError: 116828:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 2253.195842] LNet: Removed LNI 192.168.201.17@tcp2 [ 2253.250917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2253.339461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2253.373439] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2254.235474] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2254.252709] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2255.236768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2255.293561] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 2255.359737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2255.558841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2255.579452] LNet: 117113:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2255.584797] LNet: 117113:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 2255.589612] LNetError: 117113:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2255.594422] LNet: Removed LNI 192.168.201.17@tcp [ 2256.619709] LNet: Removed LNI 192.168.201.17@tcp2 [ 2256.680516] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2256.794115] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2256.812700] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2256.833695] LNet: 117119:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2257.674309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2257.882399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2257.907768] LNet: Removed LNI 192.168.201.17@tcp [ 2258.984109] LNet: Removed LNI 192.168.201.17@tcp2 [ 2259.029748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2259.112574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2259.152884] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2259.797253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2259.819181] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2260.634251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2260.682955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2260.869867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2260.897944] LNetError: 117435:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2260.903879] LNetError: 117435:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 2261.919309] LNet: Removed LNI 192.168.201.17@tcp [ 2261.929691] LNet: Removed LNI 192.168.201.17@tcp2 [ 2261.962673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2262.040564] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2262.046702] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2262.054761] LNet: 117441:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2262.590324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2262.818514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2263.839230] LNet: Removed LNI 192.168.201.17@tcp [ 2263.852356] LNet: Removed LNI 192.168.201.17@tcp2 [ 2263.920665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2263.991374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2264.035875] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2264.813349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2264.838630] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2265.555737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2265.585854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 2265.651231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2265.909315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2265.925775] LNet: 117763:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2265.931461] LNet: 117763:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 2266.975173] LNet: Removed LNI 192.168.201.17@tcp [ 2268.009698] LNet: Removed LNI 192.168.201.17@tcp2 [ 2268.044581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2268.158804] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2268.162804] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2268.167745] LNet: 117769:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2268.637148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2268.820537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2268.836710] LNet: Removed LNI 192.168.201.17@tcp [ 2269.860812] LNet: Removed LNI 192.168.201.17@tcp2 [ 2269.906619] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2270.007768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2270.053661] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2270.792603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2270.807469] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2271.621906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2271.649058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2271.698789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2272.051626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2272.096106] LNetError: 118103:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2272.099954] LNetError: 118103:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 7 previous similar messages [ 2272.107245] LNet: Removed LNI 192.168.201.17@tcp [ 2272.121062] LNet: Removed LNI 192.168.201.17@tcp2 [ 2272.143359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2272.189513] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2272.196743] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2272.202655] LNet: 118109:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2272.604337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2272.930472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2273.951339] LNet: Removed LNI 192.168.201.17@tcp [ 2273.967675] LNet: Removed LNI 192.168.201.17@tcp2 [ 2273.995341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2274.066789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2274.113744] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2275.087537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2275.116746] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2276.109624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2276.164102] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 2276.198563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 2276.259270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2276.616620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2277.663290] LNet: Removed LNI 192.168.201.17@tcp [ 2278.718425] LNet: Removed LNI 192.168.201.17@tcp2 [ 2278.763966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2278.846248] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2278.862856] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2279.349851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2286.557454] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_410 skipping ALWAYS excluded test 410 [ 2288.211231] Lustre: DEBUG MARKER: == sanity-lnet test 450: Check import of multiple interfaces ========================================================== 00:47:53 (1789447673) [ 2288.686748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2288.717400] LNet: 119228:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2288.722183] LNet: 119228:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 4 previous similar messages [ 2288.737529] LNetError: 119228:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2288.743178] LNetError: 119228:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 2288.750930] LNet: Removed LNI 192.168.201.17@tcp [ 2289.765885] LNet: Removed LNI 192.168.201.17@tcp2 [ 2289.819573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2290.203470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2290.253429] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2291.574408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2291.600763] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2292.426527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2292.449407] LNet: Removed LNI 10.1.2.3@tcp [ 2292.465622] LNet: Removed LNI 192.168.201.17@tcp [ 2292.486874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2292.552638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-450.yaml [ 2292.603498] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2292.612908] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2292.983456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del /tmp/sanity-lnet-450.yaml [ 2293.000646] LNet: Removed LNI 192.168.201.17@tcp [ 2293.010561] LNet: Removed LNI 10.1.2.3@tcp [ 2301.726462] Lustre: DEBUG MARKER: == sanity-lnet test 475: per-operation latency stats populate and reset ========================================================== 00:48:07 (1789447687) [ 2302.107996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2302.181634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2302.244602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2302.267031] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2303.006233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2303.048153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 2303.096907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2303.159322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2303.221567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2303.273522] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2303.321827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2303.374257] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2303.422671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2303.455701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2303.506521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2303.540507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2303.702325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 2310.388421] Lustre: DEBUG MARKER: == sanity-lnet test 476: lnetctl set latency_stats toggles capture ========================================================== 00:48:15 (1789447695) [ 2310.528623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2311.591964] LNet: Removed LNI 192.168.201.17@tcp [ 2311.621198] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2311.701199] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2311.749363] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2312.631980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 0 [ 2312.655787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 2312.688633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2312.717742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2312.744326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2312.775763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2312.800429] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2312.823701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2312.849395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2312.873081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2312.898712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2312.928509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2313.001959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2313.041381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2313.070671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2313.099783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2313.126108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2313.152983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2313.182680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2313.212205] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2313.243701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2313.273584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2313.304508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.17@tcp [ 2320.848091] Lustre: DEBUG MARKER: == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 00:48:26 (1789447706) [ 2321.033096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2321.076483] LNet: 121597:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2321.090668] LNet: 121597:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 2321.113929] LNetError: 121597:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2321.122467] LNetError: 121597:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 4 previous similar messages [ 2321.157833] LNet: Removed LNI 192.168.201.17@tcp [ 2321.203323] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2321.313358] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 2321.576346] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2321.580274] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2321.587958] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 2324.019781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2324.067761] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2324.908854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2324.928881] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2327.044874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2327.125517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d *@tcp -r 1 -l 1 -m PUT [ 2327.229664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.201.17@tcp -m PUT -e local_timeout -r 1 [ 2327.334263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 10.1.2.3@tcp -m PUT -e local_timeout -r 1 [ 2327.450794] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2327.459357] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2327.882991] LNetError: 121663:0:(lib-move.c:4794:lnet_parse()) 192.168.201.17@tcp, src 192.168.201.17@tcp: Dropping PUT (error -108 looking up sender) [ 2328.934424] LNet: Removed LNI 10.1.2.3@tcp [ 2329.962168] LNet: Removed LNI 192.168.201.17@tcp [ 2330.495897] Key type .llcrypt unregistered [ 2330.499503] Key type ._llcrypt unregistered [ 2344.146386] Key type ._llcrypt registered [ 2344.148737] Key type .llcrypt registered [ 2345.561639] Lustre: DEBUG MARKER: == sanity-lnet test 501: Verify lnetctl peer set --health (SR) ========================================================== 00:48:50 (1789447730) [ 2345.932338] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2345.970235] alg: No test for adler32 (adler32-zlib) [ 2346.888344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2347.504491] LNet: 122972:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2348.192305] Key type .llcrypt unregistered [ 2348.194525] Key type ._llcrypt unregistered [ 2355.159496] Key type ._llcrypt registered [ 2355.168964] Key type .llcrypt registered [ 2355.485891] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2355.517060] alg: No test for adler32 (adler32-zlib) [ 2356.529922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2356.582436] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2364.987807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 2365.198986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 2365.707244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.117@tcp [ 2365.799121] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.117@tcp [ 2365.935466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2365.957215] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2366.003983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2366.044458] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2366.095243] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2366.136690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2366.165566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2366.215814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2366.254787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2366.279114] LNet: 123421:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2366.299314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2366.339644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2366.375984] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2366.410690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2366.448771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2366.476385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2367.074746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 2367.400329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2367.443473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2367.475852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2367.505917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2367.528801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2367.576392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2367.611932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2367.646553] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2367.686934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2367.703442] LNet: 123540:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2367.727528] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2367.758963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2367.792953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2367.824333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2367.854906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2367.931318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2368.760749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.117@tcp [ 2369.027792] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2369.061108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2369.091181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2369.114912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2369.168493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2369.231920] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2369.272868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2369.327085] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2369.358611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2369.379900] LNet: 123659:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2369.400402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2369.428385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2369.465473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2369.495261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2369.537608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2369.566917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2370.323326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 2370.576476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2370.601892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2370.652731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2370.695631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2370.757894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2370.826089] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2370.875633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2370.919506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2370.948328] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2370.996229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2371.020916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2371.054512] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2371.090840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2371.135329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2371.175568] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2371.532459] LNet: 123825:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2371.541182] LNetError: 123825:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2371.562977] LNet: Removed LNI 192.168.201.17@tcp [ 2372.033432] Key type .llcrypt unregistered [ 2372.036526] Key type ._llcrypt unregistered [ 2389.307226] Key type ._llcrypt registered [ 2389.309696] Key type .llcrypt registered [ 2390.757184] Lustre: DEBUG MARKER: == sanity-lnet test 502: Verify lnetctl peer set --health (MR) ========================================================== 00:49:36 (1789447776) [ 2391.251502] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2391.283727] alg: No test for adler32 (adler32-zlib) [ 2392.325690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2392.694523] LNet: 124711:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2393.509969] Key type .llcrypt unregistered [ 2393.519107] Key type ._llcrypt unregistered [ 2400.829347] Key type ._llcrypt registered [ 2400.831980] Key type .llcrypt registered [ 2401.161346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2401.191730] alg: No test for adler32 (adler32-zlib) [ 2402.207118] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2402.237922] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2409.992498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.117@tcp [ 2418.047786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2418.103350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2418.135326] LNet: Added LNI 192.168.201.17@tcp1 [8/256/0/180] [ 2419.129674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 2419.689481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.117@tcp [ 2419.916530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2419.947449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2419.974684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2420.005963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2420.037324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2420.071718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2420.098860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2420.122757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2420.149965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2420.162791] LNet: 125363:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2420.175799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2420.197791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2420.218618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2420.244457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2420.278943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2420.317683] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2420.961960] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 2421.307572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2421.356446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2421.404415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2421.463778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2421.506412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2421.552881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2421.613753] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2421.677498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2421.733238] LNet: 2 peer NIs in recovery (showing 2): 192.168.201.117@tcp, 192.168.201.117@tcp1 [ 2421.744558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2421.781670] LNet: 125482:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2421.807403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2421.844126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2421.871623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2421.898111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2421.926969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2421.984988] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2422.766583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.117@tcp [ 2423.036538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2423.069109] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2423.098452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2423.131273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2423.167228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2423.196425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2423.242650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2423.270927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2423.306322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2423.333926] LNet: 125601:0:(api-ni.c:301:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2423.365783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2423.394832] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2423.430316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2423.459669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2423.546213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2423.622757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2424.354260] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 2424.596470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2424.627731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2424.648589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2424.675619] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2424.703910] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set latency_stats 1 [ 2424.736074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2424.759231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2424.792247] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2424.822910] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2424.850939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2424.895480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2424.930669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2424.968819] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2425.003794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2425.025644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2429.718165] LNet: 125801:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2429.721593] LNetError: 125801:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2429.724701] LNet: Removed LNI 192.168.201.17@tcp [ 2429.743562] LNet: Removed LNI 192.168.201.17@tcp1 [ 2430.152427] Key type .llcrypt unregistered [ 2430.159287] Key type ._llcrypt unregistered [ 2445.485574] Key type ._llcrypt registered [ 2445.487550] Key type .llcrypt registered [ 2446.799242] Lustre: DEBUG MARKER: == sanity-lnet test 525: keep reachable peer on local net delete ========================================================== 00:50:32 (1789447832) [ 2447.256952] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2447.283209] alg: No test for adler32 (adler32-zlib) [ 2448.178885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2448.211682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2448.225959] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2449.074650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2449.094137] LNet: Added LNI 192.168.201.17@tcp2 [8/256/0/180] [ 2449.862806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.17@tcp [ 2449.926449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp [ 2450.975490] LNet: Removed LNI 192.168.201.17@tcp [ 2457.207275] Lustre: DEBUG MARKER: == sanity-lnet test 550: DD-off: keep route up when gw reachable via another local NI ========================================================== 00:50:42 (1789447842) [ 2459.609581] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_550 Need at least 2 local interfaces for multi-rail [ 2461.178647] Lustre: DEBUG MARKER: == sanity-lnet test 600: lnet_legacy2yaml: basic ip2nets conversion ========================================================== 00:50:46 (1789447846) [ 2469.357803] Lustre: DEBUG MARKER: == sanity-lnet test 601: lnet_legacy2yaml: basic routes conversion ========================================================== 00:50:54 (1789447854) [ 2477.234582] Lustre: DEBUG MARKER: == sanity-lnet test 602: lnet_legacy2yaml: routes with priority ========================================================== 00:51:02 (1789447862) [ 2485.084775] Lustre: DEBUG MARKER: == sanity-lnet test 603: lnet_legacy2yaml: default hop flag ========================================================== 00:51:10 (1789447870) [ 2493.778931] Lustre: DEBUG MARKER: == sanity-lnet test 604: lnet_legacy2yaml: default hop with explicit value ========================================================== 00:51:18 (1789447878) [ 2502.493736] Lustre: DEBUG MARKER: == sanity-lnet test 605: lnet_legacy2yaml: quote leading star in IP pattern ========================================================== 00:51:27 (1789447887) [ 2509.764598] Lustre: DEBUG MARKER: == sanity-lnet test 606: lnet_legacy2yaml: file inputs === 00:51:35 (1789447895) [ 2516.992810] Lustre: DEBUG MARKER: == sanity-lnet test 607: lnet_legacy2yaml: options line with combined ip2nets and routes ========================================================== 00:51:42 (1789447902) [ 2526.497589] Lustre: DEBUG MARKER: == sanity-lnet test 608: lnet_legacy2yaml: error on invalid rule (missing IP pattern) ========================================================== 00:51:51 (1789447911) [ 2533.361958] Lustre: DEBUG MARKER: == sanity-lnet test 609: lnet_legacy2yaml: error on no input ========================================================== 00:51:59 (1789447919) [ 2540.240660] Lustre: DEBUG MARKER: == sanity-lnet test 610: lnet_legacy2yaml: multiple files and inline, check order ========================================================== 00:52:05 (1789447925) [ 2548.234589] Lustre: DEBUG MARKER: == sanity-lnet test 611: lnet_legacy2yaml: invalid hopcount ========================================================== 00:52:13 (1789447933) [ 2555.788447] Lustre: DEBUG MARKER: == sanity-lnet test 612: lnet_legacy2yaml: malformed IP pattern ========================================================== 00:52:21 (1789447941) [ 2563.255721] Lustre: DEBUG MARKER: == sanity-lnet test 613: lnet_legacy2yaml: multiple IP patterns in single rule ========================================================== 00:52:28 (1789447948) [ 2571.568282] Lustre: DEBUG MARKER: == sanity-lnet test 614: lnet_legacy2yaml: interfaces enumeration with multiple IP ranges ========================================================== 00:52:37 (1789447957) [ 2578.699097] Lustre: DEBUG MARKER: == sanity-lnet test 615: lnet_legacy2yaml: modprobe configuration file input ========================================================== 00:52:44 (1789447964) [ 2587.784283] Lustre: DEBUG MARKER: == sanity-lnet test 616: lnet_legacy2yaml: combining modprobe file with explicit ip2nets file ========================================================== 00:52:52 (1789447972) [ 2595.963755] Lustre: DEBUG MARKER: == sanity-lnet test 617: lnet_legacy2yaml: round-trip tunables from LNet to modprobe to YAML ========================================================== 00:53:01 (1789447981) [ 2596.053308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2596.092235] LNet: 137487:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2596.097734] LNetError: 137487:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2596.123192] LNet: Removed LNI 192.168.201.17@tcp2 [ 2596.178803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2596.348773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2596.390218] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2597.462172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2597.492544] LNet: 137635:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2597.501405] LNetError: 137635:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2597.515820] LNet: Removed LNI 192.168.201.17@tcp [ 2597.544851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2597.650915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 2597.679895] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2605.092592] Lustre: DEBUG MARKER: == sanity-lnet test 618: lnet_legacy2yaml: alternative route format ========================================================== 00:53:10 (1789447990) [ 2613.237265] Lustre: DEBUG MARKER: == sanity-lnet test 619: lnet_legacy2yaml: multiple alternative format routes from file ========================================================== 00:53:18 (1789447998) [ 2622.042441] Lustre: DEBUG MARKER: == sanity-lnet test 620: lnet_legacy2yaml: mixed legacy and alternative route formats ========================================================== 00:53:27 (1789448007) [ 2630.182614] Lustre: DEBUG MARKER: == sanity-lnet test 621: lnet_legacy2yaml: alternative format with invalid hopcount ========================================================== 00:53:35 (1789448015) [ 2638.026304] Lustre: DEBUG MARKER: == sanity-lnet test 622: lnet_legacy2yaml: alternative format with default-hop flag ========================================================== 00:53:43 (1789448023) [ 2645.993938] Lustre: DEBUG MARKER: == sanity-lnet test 623: lnet_legacy2yaml: basic networks conversion ========================================================== 00:53:51 (1789448031) [ 2653.783619] Lustre: DEBUG MARKER: == sanity-lnet test 624: lnet_legacy2yaml: networks with interfaces ========================================================== 00:53:58 (1789448038) [ 2660.548316] Lustre: DEBUG MARKER: == sanity-lnet test 625: lnet_legacy2yaml: multiple networks ========================================================== 00:54:06 (1789448046) [ 2667.780774] Lustre: DEBUG MARKER: == sanity-lnet test 626: lnet_legacy2yaml: mixed networks, ip2nets, and routes ========================================================== 00:54:13 (1789448053) [ 2676.048613] Lustre: DEBUG MARKER: == sanity-lnet test 627: lnet_legacy2yaml: invalid networks format ========================================================== 00:54:21 (1789448061) [ 2683.929922] Lustre: DEBUG MARKER: == sanity-lnet test 628: lnet_legacy2yaml: missing delimiter error ========================================================== 00:54:29 (1789448069) [ 2691.327540] Lustre: DEBUG MARKER: == sanity-lnet test 629: lnet_legacy2yaml: multiple gateways error ========================================================== 00:54:36 (1789448076) [ 2698.144577] Lustre: DEBUG MARKER: == sanity-lnet test complete, duration 2426 sec ========== 00:54:43 (1789448083) [ 2698.434331] LNet: 145099:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2698.439524] LNetError: 145099:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2698.455497] LNet: Removed LNI 192.168.201.17@tcp [ 2699.241784] Key type .llcrypt unregistered [ 2699.245297] Key type ._llcrypt unregistered [ 2711.009077] Key type ._llcrypt registered [ 2711.012437] Key type .llcrypt registered [ 2711.477544] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2711.495632] alg: No test for adler32 (adler32-zlib) [ 2712.540830] Lustre: Lustre: Build Version: 2.17.58_39_ge07ba0b [ 2712.786512] LNet: Added LNI 192.168.201.17@tcp [8/256/0/180] [ 2714.503259] Key type lgssc registered [ 2715.973263] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2783.930143] Lustre: Mounted lustre-client - version 2.17.58_39_ge07ba0b [ 2788.698823] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2796.132273] Lustre: Unmounted lustre-client [ 2813.740859] Key type lgssc unregistered [ 2814.094848] LNet: 148044:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2814.099839] LNetError: 148044:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2814.118914] LNet: Removed LNI 192.168.201.17@tcp [ 2814.912357] Key type .llcrypt unregistered [ 2814.913637] Key type ._llcrypt unregistered