[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 428454775 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54d0-0x000f54df] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2576MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829700K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.002372] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004016] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.007021] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.008015] pid_max: default: 32768 minimum: 301 [ 0.009140] LSM: Security Framework initializing [ 0.011029] Yama: becoming mindful. [ 0.012046] SELinux: Initializing. [ 0.013075] *** VALIDATE selinux *** [ 0.021843] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026470] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027159] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028113] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030038] *** VALIDATE tmpfs *** [ 0.031482] *** VALIDATE proc *** [ 0.033193] *** VALIDATE cgroup *** [ 0.034010] *** VALIDATE cgroup2 *** [ 0.035269] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036166] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038029] Spectre V2 : User space: Vulnerable [ 0.039009] Speculative Store Bypass: Vulnerable [ 0.042203] debug: unmapping init [mem 0xffffffffabe59000-0xffffffffabe60fff] [ 0.045244] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046809] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047037] ... version: 2 [ 0.048014] ... bit width: 48 [ 0.049013] ... generic registers: 4 [ 0.050014] ... value mask: 0000ffffffffffff [ 0.051014] ... max period: 00007fffffffffff [ 0.052015] ... fixed-purpose events: 3 [ 0.053013] ... event mask: 000000070000000f [ 0.054311] rcu: Hierarchical SRCU implementation. [ 0.056529] smp: Bringing up secondary CPUs ... [ 0.057675] x86: Booting SMP configuration: [ 0.058025] .... node #0, CPUs: #1 #2 #3 [ 0.063156] smp: Brought up 1 node, 4 CPUs [ 0.065014] smpboot: Max logical packages: 1 [ 0.066012] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.149662] node 0 deferred pages initialised in 81ms [ 0.152434] devtmpfs: initialized [ 0.153149] x86/mm: Memory block size: 128MB [ 0.155636] gcov: version magic: 0x41383552 [ 0.156685] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.160126] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.162352] pinctrl core: initialized pinctrl subsystem [ 0.164129] [ 0.164494] ************************************************************* [ 0.166015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.168012] ** ** [ 0.170016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.172014] ** ** [ 0.175012] ** This means that this kernel is built to expose internal ** [ 0.177011] ** IOMMU data structures, which may compromise security on ** [ 0.179015] ** your system. ** [ 0.181014] ** ** [ 0.183012] ** If you see this message and you are not debugging the ** [ 0.185014] ** kernel, report this immediately to your vendor! ** [ 0.187013] ** ** [ 0.190015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.192011] ************************************************************* [ 0.194881] NET: Registered protocol family 16 [ 0.196461] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.199120] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.201063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.204112] cpuidle: using governor menu [ 0.205631] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.207595] PCI: Using configuration type 1 for base access [ 0.210217] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.222080] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.223036] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.225071] cryptd: max_cpu_qlen set to 1000 [ 0.229216] ACPI: Added _OSI(Module Device) [ 0.231017] ACPI: Added _OSI(Processor Device) [ 0.232014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.234014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.238964] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.244525] ACPI: Interpreter enabled [ 0.246064] ACPI: PM: (supports S0 S3 S4 S5) [ 0.247012] ACPI: Using IOAPIC for interrupt routing [ 0.249097] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.251357] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.262625] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.265046] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.268025] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.272081] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.276261] acpiphp: Slot [2] registered [ 0.278103] acpiphp: Slot [5] registered [ 0.279114] acpiphp: Slot [6] registered [ 0.280153] acpiphp: Slot [3] registered [ 0.281079] acpiphp: Slot [4] registered [ 0.282092] acpiphp: Slot [7] registered [ 0.284096] acpiphp: Slot [8] registered [ 0.285111] acpiphp: Slot [9] registered [ 0.286110] acpiphp: Slot [10] registered [ 0.288093] acpiphp: Slot [11] registered [ 0.289093] acpiphp: Slot [12] registered [ 0.290061] acpiphp: Slot [13] registered [ 0.290872] acpiphp: Slot [14] registered [ 0.292044] acpiphp: Slot [15] registered [ 0.293060] acpiphp: Slot [16] registered [ 0.294086] acpiphp: Slot [17] registered [ 0.294929] acpiphp: Slot [18] registered [ 0.295050] acpiphp: Slot [19] registered [ 0.296142] acpiphp: Slot [20] registered [ 0.298107] acpiphp: Slot [21] registered [ 0.299088] acpiphp: Slot [22] registered [ 0.300081] acpiphp: Slot [23] registered [ 0.301077] acpiphp: Slot [24] registered [ 0.303077] acpiphp: Slot [25] registered [ 0.304084] acpiphp: Slot [26] registered [ 0.306077] acpiphp: Slot [27] registered [ 0.307094] acpiphp: Slot [28] registered [ 0.309127] acpiphp: Slot [29] registered [ 0.310165] acpiphp: Slot [30] registered [ 0.312110] acpiphp: Slot [31] registered [ 0.313146] PCI host bridge to bus 0000:00 [ 0.315019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.316022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.318018] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.321012] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.323028] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.326030] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.328168] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.331011] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.334304] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.341589] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.346114] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.348019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.350015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.352020] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.354584] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.358199] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.361048] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.365023] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.370017] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.379763] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.384030] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.390537] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.401016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.410020] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.423018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.432558] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.438015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.442014] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.453014] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.462513] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.464379] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.467368] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.470411] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.472231] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.477086] iommu: Default domain type: Passthrough [ 0.479444] SCSI subsystem initialized [ 0.481136] ACPI: bus type USB registered [ 0.482119] usbcore: registered new interface driver usbfs [ 0.484090] usbcore: registered new interface driver hub [ 0.486077] usbcore: registered new device driver usb [ 0.488163] pps_core: LinuxPPS API ver. 1 registered [ 0.489010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.492058] PTP clock support registered [ 0.494106] EDAC MC: Ver: 3.0.0 [ 0.496361] PCI: Using ACPI for IRQ routing [ 0.498719] NetLabel: Initializing [ 0.500011] NetLabel: domain hash size = 128 [ 0.501009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.503084] NetLabel: unlabeled traffic allowed by default [ 0.505184] vgaarb: loaded [ 0.507256] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.509018] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.514284] clocksource: Switched to clocksource kvm-clock [ 0.622903] VFS: Disk quotas dquot_6.6.0 [ 0.624625] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.627147] *** VALIDATE ramfs *** [ 0.628441] *** VALIDATE hugetlbfs *** [ 0.629995] pnp: PnP ACPI init [ 0.632336] pnp: PnP ACPI: found 6 devices [ 0.652385] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.655811] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.657484] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.659573] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.662040] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.664287] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.666985] NET: Registered protocol family 2 [ 0.669402] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.674244] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.677974] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.683203] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.686867] TCP: Hash tables configured (established 65536 bind 65536) [ 0.689932] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.692776] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.695298] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.697443] NET: Registered protocol family 1 [ 0.699528] RPC: Registered named UNIX socket transport module. [ 0.701549] RPC: Registered udp transport module. [ 0.702939] RPC: Registered tcp transport module. [ 0.704169] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.705711] NET: Registered protocol family 44 [ 0.707705] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.710328] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.712872] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.715514] PCI: CLS 0 bytes, default 64 [ 0.717388] Unpacking initramfs... [ 2.067458] debug: unmapping init [mem 0xffff98cf3cc64000-0xffff98cf3ffcffff] [ 2.069828] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.071052] software IO TLB: mapped [mem 0x00000000b8c64000-0x00000000bcc64000] (64MB) [ 2.072581] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 2.552308] Initialise system trusted keyrings [ 2.554136] Key type blacklist registered [ 2.555815] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.600587] zbud: loaded [ 2.604389] *** VALIDATE nfs *** [ 2.605367] *** VALIDATE nfs4 *** [ 2.607317] pstore: using deflate compression [ 2.611427] Platform Keyring initialized [ 2.707517] NET: Registered protocol family 38 [ 2.709354] Key type asymmetric registered [ 2.710505] Asymmetric key parser 'x509' registered [ 2.711941] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.714366] io scheduler mq-deadline registered [ 2.715833] io scheduler kyber registered [ 2.717157] io scheduler bfq registered [ 2.718658] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.721443] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.723851] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.726478] ACPI: Power Button [PWRF] [ 2.731786] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.738612] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.748286] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.776242] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.805847] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.811217] Non-volatile memory driver v1.3 [ 2.812836] Linux agpgart interface v0.103 [ 2.843086] virtio_blk virtio1: [vda] 147160 512-byte logical blocks (75.3 MB/71.9 MiB) [ 2.846017] vda: detected capacity change from 0 to 75345920 [ 2.868449] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.871264] vdb: detected capacity change from 0 to 1073741824 [ 2.886275] libphy: Fixed MDIO Bus: probed [ 2.894305] usbcore: registered new interface driver usbserial_generic [ 2.896625] usbserial: USB Serial support registered for generic [ 2.899260] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.903477] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.905397] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.907442] mousedev: PS/2 mouse device common for all mice [ 2.910281] rtc_cmos 00:05: RTC can wake from S4 [ 2.912726] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.913744] rtc_cmos 00:05: registered as rtc0 [ 2.918960] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.919119] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.924785] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.925038] intel_pstate: CPU model not supported [ 2.931331] hid: raw HID events driver (C) Jiri Kosina [ 2.933296] usbcore: registered new interface driver usbhid [ 2.935222] usbhid: USB HID core driver [ 2.936697] drop_monitor: Initializing network drop monitor service [ 2.939459] Initializing XFRM netlink socket [ 2.941454] NET: Registered protocol family 10 [ 2.944913] Segment Routing with IPv6 [ 2.946528] NET: Registered protocol family 17 [ 2.949310] mpls_gso: MPLS GSO support [ 2.954651] RAS: Correctable Errors collector initialized. [ 2.957055] AVX version of gcm_enc/dec engaged. [ 2.958916] AES CTR mode by8 optimization enabled [ 3.037195] sched_clock: Marking stable (3037164661, 0)->(3901881926, -864717265) [ 3.041288] registered taskstats version 1 [ 3.043531] Loading compiled-in X.509 certificates [ 3.045580] zswap: loaded using pool lzo/zbud [ 3.071799] Key type big_key registered [ 3.086051] Key type encrypted registered [ 3.087534] ima: No TPM chip found, activating TPM-bypass! [ 3.089592] ima: Allocated hash algorithm: sha1 [ 3.091193] ima: No architecture policies found [ 3.092975] evm: Initialising EVM extended attributes: [ 3.094910] evm: security.selinux [ 3.096138] evm: security.ima [ 3.097295] evm: security.capability [ 3.098635] evm: HMAC attrs: 0x1 [ 3.102634] rtc_cmos 00:05: setting system clock to 2026-09-11 13:06:49 UTC (1789132009) [ 3.108989] debug: unmapping init [mem 0xfffffffface03000-0xffffffffacffffff] [ 3.112070] debug: unmapping init [mem 0xffffffffabb82000-0xffffffffabe58fff] [ 3.122120] Write protecting the kernel read-only data: 28672k [ 3.125438] debug: unmapping init [mem 0xffffffffaa203000-0xffffffffaa3fffff] [ 3.127564] debug: unmapping init [mem 0xffffffffaab14000-0xffffffffaabfffff] [ 3.170731] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.178834] systemd[1]: Detected virtualization kvm. [ 3.180711] systemd[1]: Detected architecture x86-64. [ 3.182714] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.210630] systemd[1]: No hostname configured. [ 3.212664] systemd[1]: Set hostname to . [ 3.214720] random: systemd: uninitialized urandom read (16 bytes read) [ 3.217136] systemd[1]: Initializing machine ID from random generator. [ 3.393586] random: systemd: uninitialized urandom read (16 bytes read) [ 3.395864] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.399617] random: systemd: uninitialized urandom read (16 bytes read) [ 3.402708] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.408194] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Swap. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.068651] device-mapper: uevent: version 1.0.3 [ 4.071233] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.788898] virtio_net virtio0 ens2: renamed from eth0 [ 4.907067] scsi host0: ata_piix [ 4.909415] scsi host1: ata_piix [ 4.911047] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 4.913324] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 9.088157] dracut-initqueue[590]: RTNETLINK answers: File exists [ 9.605912] random: crng init done [ 9.607106] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 9.965740] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.039096] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.292584] SELinux: Disabled at runtime. [ 11.353344] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.361536] systemd[1]: Detected virtualization kvm. [ 11.363268] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.910609] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.913896] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.918289] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.921567] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.924656] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.932469] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.941625] systemd[1]: Activating swap /dev/disk/by-label/SWAP... Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Starting Remount Root and Kernel File Systems... [ OK ] Created slice User and Session Slice. [ 11.975710] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-serial\x2dgetty.slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-getty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. Mounting POSIX Message Queue File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Huge Pages File System... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Kernel Debug File System... [ OK ] Stopped target Initrd Root File System. Starting Apply Kernel Variables... [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.377071] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.581732] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.641931] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.736267] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.753066] EDAC sbridge: Ver: 1.1.2 [ 13.863134] Key type dns_resolver registered [ 14.157045] NFS: Registering the id_resolver key type [ 14.158598] Key type id_resolver registered [ 14.159699] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg155-client login: [ 72.176343] libcfs: loading out-of-tree module taints kernel. [ 72.502349] Key type ._llcrypt registered [ 72.507849] Key type .llcrypt registered [ 73.237053] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 73.278639] alg: No test for adler32 (adler32-zlib) [ 74.910402] Lustre: Lustre: Build Version: 2.17.57_131_gc13dec9 [ 76.337206] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 76.695180] hrtimer: interrupt took 17169837 ns [ 78.288533] Key type lgssc registered [ 80.745134] Lustre: Echo OBD driver; http://www.lustre.org/ [ 239.601411] Lustre: Mounted lustre-client - version 2.17.57_131_gc13dec9 [ 246.535403] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 262.986104] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing check_logdir /tmp/testlogs/ [ 265.184296] Lustre: lustre-OST0000-osc-ffff98cf91e0f000: disconnect after 22s idle [ 271.376977] Lustre: DEBUG MARKER: oleg155-client.virtnet: executing yml_node [ 276.595761] Lustre: DEBUG MARKER: Client: 2.17.57.131 [ 279.661042] Lustre: DEBUG MARKER: MDS: 2.17.57.131 [ 283.136881] Lustre: DEBUG MARKER: OSS: 2.17.57.131 [ 285.482931] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Fri Sep 11 09:11:29 EDT 2026 [ 305.264256] Lustre: Unmounted lustre-client [ 332.553811] Key type lgssc unregistered [ 333.026220] LNet: 8434:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 333.041130] LNetError: 8434:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 333.083693] LNet: Removed LNI 192.168.201.55@tcp [ 334.430258] Key type .llcrypt unregistered [ 334.437273] Key type ._llcrypt unregistered [ 369.221733] Key type ._llcrypt registered [ 369.230516] Key type .llcrypt registered [ 369.484305] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 369.507902] alg: No test for adler32 (adler32-zlib) [ 370.525397] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 370.640327] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 370.684893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show [ 376.889629] Lustre: DEBUG MARKER: excepting tests: 253 254 410 [ 377.181157] LNet: 9319:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 377.196545] LNetError: 9319:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 377.239545] LNet: Removed LNI 192.168.201.55@tcp [ 378.267672] Key type .llcrypt unregistered [ 378.272779] Key type ._llcrypt unregistered [ 387.434435] Key type ._llcrypt registered [ 387.436718] Key type .llcrypt registered [ 389.101386] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 09:13:13 (1789132393) [ 389.885700] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 389.917566] alg: No test for adler32 (adler32-zlib) [ 390.889812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 391.043562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 391.093704] LNet: 9739:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 391.141735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 391.201649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 391.248264] LNet: 9746:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 399.377831] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 09:13:24 (1789132404) [ 399.767534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 399.814579] LNet: 10324:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 399.851531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 399.969233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp [ 408.332230] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 09:13:33 (1789132413) [ 408.759425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 408.795617] LNet: 10910:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 408.833780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 408.967460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 2.2.2.2@o2ib [ 417.275227] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 09:13:42 (1789132422) [ 417.685342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 417.703880] LNet: 11494:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 417.739120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 417.876521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib [ 427.099719] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 09:13:51 (1789132431) [ 427.290610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 427.335466] LNet: 12076:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 427.372290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 427.575418] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp [ 428.236714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp [ 436.254225] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 09:14:00 (1789132440) [ 436.482290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 436.517257] LNet: 12672:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 436.540609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 436.611279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 436.931384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 437.495630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib [ 445.040772] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 09:14:10 (1789132450) [ 445.273033] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 445.328073] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 445.466200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi,[6-12/4]@bxi3f [ 452.600870] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 09:14:17 (1789132457) [ 452.942507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 452.976966] LNet: 13853:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 452.982340] LNet: 13853:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 453.019642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 453.109681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp [ 453.147837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 453.503274] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib [ 453.563719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 454.007622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp [ 454.076725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 454.565079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib [ 454.626641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 455.269743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib [ 455.317878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 455.794659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni [ 455.831523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 456.430577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni [ 456.478668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 457.161178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi [ 457.220563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 457.764533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi [ 457.801869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 458.244800] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f [ 458.280767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 458.838289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f --nid [8-12]@bxi3f [ 458.889063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 459.682611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi,[3-7]@bxi3f [ 459.743523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 460.506867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 460.550976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 469.235121] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 09:14:33 (1789132473) [ 469.395355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 469.465835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 469.600306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp [ 469.653614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp [ 477.101748] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 09:14:42 (1789132482) [ 477.199095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 477.262564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 477.373345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 477.429740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 485.840996] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 09:14:50 (1789132490) [ 485.946814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 486.009824] LNet: 15704:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 486.012268] LNet: 15704:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 486.052300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 486.186191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp [ 486.225361] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp [ 493.954655] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 09:14:59 (1789132499) [ 494.481279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 494.558193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 494.681309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 494.734563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 505.586424] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 09:15:09 (1789132509) [ 505.704417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 505.771304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 505.909806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp [ 505.962344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp [ 514.700468] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 09:15:19 (1789132519) [ 514.902112] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 515.055414] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 515.220759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 515.301154] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 524.897509] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 09:15:29 (1789132529) [ 525.346681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 525.382955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 525.538603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 526.209949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 534.839403] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 09:15:39 (1789132539) [ 535.022634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 535.111958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 535.485701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 535.867760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 545.256803] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 09:15:49 (1789132549) [ 545.391240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 545.432604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 545.668914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 546.059375] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 554.382295] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 09:15:59 (1789132559) [ 554.666915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 554.692892] LNet: 19888:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 554.696115] LNet: 19888:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 6 previous similar messages [ 554.709222] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 555.134587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 555.489748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 563.764862] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 09:16:08 (1789132568) [ 563.969361] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 564.091778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 564.236743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 564.357968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 572.726871] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 09:16:17 (1789132577) [ 573.042575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 573.105401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 573.213512] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 573.277730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 580.987925] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 09:16:25 (1789132585) [ 581.139257] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 581.230691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 581.333658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 19@gni [ 588.637932] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 09:16:33 (1789132593) [ 588.771145] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 588.803836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 588.909562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib [ 596.884518] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 09:16:41 (1789132601) [ 597.142190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 597.198255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 597.259935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni [ 597.587926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni [ 605.441423] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 09:16:50 (1789132610) [ 605.564096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 605.636591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 605.781745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni [ 605.834609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 22@gni --nid 26@gni [ 613.177518] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 09:16:57 (1789132617) [ 613.461266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 613.572238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 613.707133] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni [ 613.783648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni [ 622.280622] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 09:17:07 (1789132627) [ 622.510653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 622.585772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 622.647953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni [ 622.698937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp [ 630.806576] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 09:17:15 (1789132635) [ 630.907723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 630.992530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 631.132287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 631.194838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 639.491373] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 09:17:23 (1789132643) [ 639.649520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 639.751754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 639.855638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim [ 639.930656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp [ 639.954811] LNetError: 25861:0:(peer.c:2362:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 639.995674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --force [ 640.021705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show --nid 1.1.1.1@tcp [ 647.826836] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 09:17:32 (1789132652) [ 647.954966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 648.029470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 648.191751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.iU8hahjvw1 [ 648.313195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.iU8hahjvw1 [ 648.397833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.iU8hahjvw1 [ 659.023771] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 09:17:42 (1789132662) [ 659.269278] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 659.443796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 659.524277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp [ 659.596040] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp [ 659.640399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer list [ 669.381676] Lustre: DEBUG MARKER: == sanity-lnet test 40: LNetAddPeer merges a pre-existing non-primary peer NID ========================================================== 09:17:53 (1789132673) [ 669.508508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 669.574693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 669.783441] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 669.859785] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 672.893865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 675.879589] Lustre: Lustre: Build Version: 2.17.57_131_gc13dec9 [ 678.456179] Key type lgssc registered [ 679.363627] Lustre: Echo OBD driver; http://www.lustre.org/ [ 685.551071] LNetError: 27623:0:(lib-move.c:2158:lnet_handle_find_routed_path()) peer 192.168.201.55@tcp99 has no available nets [ 688.787368] Key type lgssc unregistered [ 689.103507] LNet: 28738:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 689.108148] LNet: 28738:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 13 previous similar messages [ 689.123645] LNetError: 28738:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 689.143935] LNet: Removed LNI 192.168.201.55@tcp [ 689.628478] Key type .llcrypt unregistered [ 689.630798] Key type ._llcrypt unregistered [ 698.270266] Key type ._llcrypt registered [ 698.275748] Key type .llcrypt registered [ 699.696939] Lustre: DEBUG MARKER: == sanity-lnet test 50: Enable/disable routing via yaml import ========================================================== 09:18:24 (1789132704) [ 700.418167] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 700.443957] alg: No test for adler32 (adler32-zlib) [ 701.520866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 701.630880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 706.017066] LNet: Message forwarding will be enabled in 2s [ 709.088370] LNet: Message forwarding enabled [ 709.962896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 710.022493] LNet: Message forwarding will stop in 2s to 6s [ 713.833839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 716.497225] LNet: Message forwarding disabled [ 718.359231] LNet: Message forwarding will be enabled in 2s [ 720.681564] LNet: Message forwarding enabled [ 728.714558] Lustre: DEBUG MARKER: == sanity-lnet test 51: Set numa range via yaml import === 09:18:53 (1789132733) [ 729.002437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 729.049535] LNet: 30139:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 731.458891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 731.569515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-51.yaml [ 740.828255] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 09:19:05 (1789132745) [ 741.243341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 741.281701] LNet: 30723:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 741.299938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 741.342727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid foobar [ 741.367395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid foobar [ 741.388567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@o2ib [ 741.406247] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --nid 1.1.1.1@tcp [ 741.426942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --nid 1.1.1.1@tcp [ 741.448272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add [ 741.482492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 741.505422] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp [ 741.523675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@foo [ 741.545472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@tcp [ 741.569220] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@o2ib [ 741.589628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@gni [ 741.612424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni [ 741.636891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni [ 741.664474] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp [ 741.693493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp [ 741.719374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp [ 741.747996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 741.790393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp --gateway 1@gni [ 741.809670] LNetError: 30766:0:(router.c:625:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni [ 748.564901] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 09:19:13 (1789132753) [ 748.750612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 748.796642] LNet: 31332:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 748.839141] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 749.000734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 757.047680] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 09:19:21 (1789132761) [ 759.535452] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_100 Need at least 2 remote nodes found oleg155-server [ 762.222774] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 09:19:26 (1789132766) [ 762.345365] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 762.398257] LNet: 32263:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 762.450444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 762.602675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 762.642149] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 764.141853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp1 --gateway 192.168.201.[56-60]@tcp [ 764.185436] LNet: 32404:0:(router.c:729:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 764.268231] LNetError: 32273:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.56@tcp is being used as a gateway but routing feature is not turned on [ 764.297332] LNetError: 32273:0:(lib-lnet.h:1280:lnet_set_route_aliveness()) route to tcp1 through 192.168.201.56@tcp has gone from up to down [ 765.305062] LNetError: 32273:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.59@tcp is being used as a gateway but routing feature is not turned on [ 765.326680] LNetError: 32273:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 9 previous similar messages [ 766.343111] LNetError: 32273:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.59@tcp is being used as a gateway but routing feature is not turned on [ 766.362810] LNetError: 32273:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 4 previous similar messages [ 768.387366] LNetError: 32273:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.59@tcp is being used as a gateway but routing feature is not turned on [ 768.408775] LNetError: 32273:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 9 previous similar messages [ 772.451374] LNetError: 32273:0:(router.c:394:lnet_router_discovery_ping_reply()) Peer 192.168.201.60@tcp is being used as a gateway but routing feature is not turned on [ 772.464850] LNetError: 32273:0:(router.c:394:lnet_router_discovery_ping_reply()) Skipped 19 previous similar messages [ 772.634272] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 09:19:37 (1789132777) [ 774.628868] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_102 Need at least 2 remote nodes found oleg155-server [ 776.362461] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 09:19:41 (1789132781) [ 776.735627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 776.755687] LNet: 33340:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 776.769177] LNetError: 33340:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 776.790447] LNet: Removed LNI 192.168.201.55@tcp [ 776.820759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 776.872701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 776.898284] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 777.661612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp103 --gateway 192.168.201.[56-61/2]@tcp [ 777.696302] LNet: 33457:0:(router.c:729:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 777.704657] LNet: 33457:0:(router.c:729:lnet_add_route()) Skipped 4 previous similar messages [ 777.734209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route del --net tcp103 --gateway 192.168.201.[56-61/2]@tcp [ 784.919163] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 09:19:50 (1789132790) [ 785.283400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 785.323308] LNet: 34050:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 785.333856] LNetError: 34050:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 785.352389] LNet: Removed LNI 192.168.201.55@tcp [ 785.373748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 785.465212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking -1 [ 785.516943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 785.551867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 785.602884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 785.654389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 785.690179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 785.755611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 0 [ 785.844303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 785.883506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 785.956614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 786.052201] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 786.094876] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 786.216724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 1 [ 786.319186] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 786.363326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 786.472758] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 786.578660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 786.630599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 786.684909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 2 [ 786.794624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 786.821969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 786.897269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 786.983806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 787.052866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 787.138157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 787.232507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 787.273636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 787.333943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 787.415480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 787.455731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 787.541924] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 4 [ 787.598495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 787.637952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 787.729850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 794.987483] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 09:19:59 (1789132799) [ 796.818261] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_105 Need at least 2 remote nodes found oleg155-server [ 799.028392] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 09:20:03 (1789132803) [ 801.230914] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_106 Need at least 2 remote nodes found oleg155-server [ 803.912832] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 09:20:08 (1789132808) [ 804.263262] LNet: 35451:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 804.270226] LNet: 35451:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 11 previous similar messages [ 805.111744] Key type .llcrypt unregistered [ 805.115607] Key type ._llcrypt unregistered [ 813.201436] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 813.375592] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 813.389068] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 813.406373] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 816.229867] Key type ._llcrypt registered [ 816.232807] Key type .llcrypt registered [ 816.408811] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 816.421462] alg: No test for adler32 (adler32-zlib) [ 817.307942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 817.348731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 817.367607] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 817.833630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 817.853687] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 819.633749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 819.651780] LNet: Removed LNI 10.1.2.3@tcp [ 819.745554] LNet: 36176:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 819.751442] LNetError: 36176:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 819.765425] LNet: Removed LNI 192.168.201.55@tcp [ 820.322552] Key type .llcrypt unregistered [ 820.325871] Key type ._llcrypt unregistered [ 834.882716] Key type ._llcrypt registered [ 834.885748] Key type .llcrypt registered [ 836.955120] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 09:20:41 (1789132841) [ 837.562370] Key type .llcrypt unregistered [ 837.567894] Key type ._llcrypt unregistered [ 845.975456] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 846.158190] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 846.167473] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 846.172164] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 848.930432] Key type ._llcrypt registered [ 848.933421] Key type .llcrypt registered [ 849.135468] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 849.148643] alg: No test for adler32 (adler32-zlib) [ 850.094915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 850.172857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 850.201117] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 851.258276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 851.296901] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 854.917710] LNet: 37679:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 854.939234] LNetError: 37679:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 854.948541] LNet: Removed LNI 10.1.2.3@tcp [ 854.968597] LNet: Removed LNI 192.168.201.55@tcp [ 855.628430] Key type .llcrypt unregistered [ 855.632936] Key type ._llcrypt unregistered [ 870.823421] Key type ._llcrypt registered [ 870.826527] Key type .llcrypt registered [ 872.156624] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 09:21:17 (1789132877) [ 872.857314] Key type .llcrypt unregistered [ 872.860203] Key type ._llcrypt unregistered [ 880.801906] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 880.969094] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 880.973539] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 880.979590] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 883.947380] Key type ._llcrypt registered [ 883.949557] Key type .llcrypt registered [ 884.296861] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 884.321897] alg: No test for adler32 (adler32-zlib) [ 885.402964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 885.477866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 885.507107] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 888.310644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 888.341966] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 890.936711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 890.976338] LNet: Removed LNI 10.1.2.3@tcp [ 890.994519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg:0 [ 891.025810] LNet: Removed LNI 10.1.2.31@tcp [ 891.061860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 891.108048] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 891.954768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 891.984036] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 893.111470] LNet: 39605:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 893.119627] LNetError: 39605:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg:0 not found [ 893.125947] LNet: Removed LNI 10.1.2.31@tcp [ 893.140649] LNet: Removed LNI 10.1.2.3@tcp [ 893.938519] Key type .llcrypt unregistered [ 893.940199] Key type ._llcrypt unregistered [ 907.919307] Key type ._llcrypt registered [ 907.923463] Key type .llcrypt registered [ 909.567403] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 09:21:54 (1789132914) [ 910.346722] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 910.355557] alg: No test for adler32 (adler32-zlib) [ 911.334604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 911.384322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 911.414176] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 912.472468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 912.500249] LNet: 40566:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 912.507938] LNetError: 40566:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 912.532675] LNet: Removed LNI 192.168.201.55@tcp [ 912.552063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 912.612216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --nid 192.168.201.55@tcp [ 912.645643] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 913.390415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --nid 192.168.201.55@tcp [ 913.423640] LNet: Removed LNI 192.168.201.55@tcp [ 913.520206] LNet: 40701:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 914.441457] Key type .llcrypt unregistered [ 914.443333] Key type ._llcrypt unregistered [ 929.895720] Key type ._llcrypt registered [ 929.898740] Key type .llcrypt registered [ 931.946625] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 09:22:16 (1789132936) [ 934.337170] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_111 Need at least 2 remote nodes found oleg155-server [ 935.827773] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 09:22:21 (1789132941) [ 936.587312] Key type .llcrypt unregistered [ 936.589418] Key type ._llcrypt unregistered [ 944.919824] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 945.126517] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 945.132994] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 945.154266] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 948.146906] Key type ._llcrypt registered [ 948.148439] Key type .llcrypt registered [ 948.477726] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 948.498623] alg: No test for adler32 (adler32-zlib) [ 949.547295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 949.641650] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 950.728365] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180] [ 953.037935] LNet: 42547:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 953.042909] LNetError: 42547:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 953.049620] LNet: Removed LNI 192.168.201.55@tcp1 [ 953.058552] LNet: Removed LNI 10.1.2.3@tcp2 [ 953.749494] Key type .llcrypt unregistered [ 953.753316] Key type ._llcrypt unregistered [ 966.684571] Key type ._llcrypt registered [ 966.686839] Key type .llcrypt registered [ 968.355226] Lustre: DEBUG MARKER: == sanity-lnet test 150: Check import of ip2nets yaml sequence ========================================================== 09:22:52 (1789132972) [ 968.976719] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 969.002594] alg: No test for adler32 (adler32-zlib) [ 970.051316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 970.144280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 970.220461] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 970.983954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 971.002170] LNet: 43522:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 971.007971] LNetError: 43522:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 971.023575] LNet: Removed LNI 192.168.201.55@tcp [ 971.049339] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 971.136989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet.yaml [ 971.186693] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 979.926508] Lustre: DEBUG MARKER: == sanity-lnet test 151: Check correct application of multiple ip2nets rules ========================================================== 09:23:04 (1789132984) [ 980.044577] LNet: 44104:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 980.053270] LNetError: 44104:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 980.076694] LNet: Removed LNI 192.168.201.55@tcp [ 981.314781] Key type .llcrypt unregistered [ 981.317523] Key type ._llcrypt unregistered [ 990.553045] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 990.674319] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 990.694300] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 990.708564] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 993.616641] Key type ._llcrypt registered [ 993.619511] Key type .llcrypt registered [ 993.787268] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 993.799787] alg: No test for adler32 (adler32-zlib) [ 994.792635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 994.876633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 994.895268] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 995.634344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 995.650426] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 998.089502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 998.105816] LNet: 44863:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 998.110334] LNetError: 44863:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 998.117490] LNet: Removed LNI 10.1.2.3@tcp [ 998.132787] LNet: Removed LNI 192.168.201.55@tcp [ 998.172861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 998.244985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 998.273939] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 998.277352] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 998.839241] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 998.857607] LNet: 44890:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 998.871592] LNetError: 44890:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 998.876226] LNetError: 44890:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 998.881990] LNet: Removed LNI 10.1.2.3@tcp [ 998.891909] LNet: Removed LNI 192.168.201.55@tcp [ 998.923666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 998.981847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 999.003659] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 999.007669] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 999.542318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 999.579682] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 1000.881499] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1000.899356] LNet: 45052:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1000.904102] LNetError: 45052:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1000.908777] LNetError: 45052:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1000.916718] LNet: Removed LNI 10.1.2.3@tcp [ 1000.918501] LNet: Removed LNI 192.168.201.55@tcp [ 1000.925700] LNet: Removed LNI 192.168.201.55@tcp2 [ 1000.941576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1001.000558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 1001.024374] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1001.028182] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1001.035721] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 1010.786445] Lustre: DEBUG MARKER: == sanity-lnet test 152: Check import of ip2nets with wildcard IP patterns ========================================================== 09:23:35 (1789133015) [ 1011.165261] LNet: 45650:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1011.182547] LNetError: 45650:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1011.186066] LNetError: 45650:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 1011.208104] LNet: Removed LNI 10.1.2.3@tcp [ 1011.213027] LNet: Removed LNI 192.168.201.55@tcp [ 1011.246880] LNet: Removed LNI 192.168.201.55@tcp2 [ 1012.323469] Key type .llcrypt unregistered [ 1012.336356] Key type ._llcrypt unregistered [ 1020.810714] Key type ._llcrypt registered [ 1020.813807] Key type .llcrypt registered [ 1021.193379] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1021.234530] alg: No test for adler32 (adler32-zlib) [ 1022.396931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1022.559957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1022.607311] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1023.639481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1023.659392] LNet: 46050:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1023.665418] LNetError: 46050:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1023.685452] LNet: Removed LNI 192.168.201.55@tcp [ 1023.706992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1023.809322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1023.866843] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1024.315258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1024.331238] LNet: 46080:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1024.336472] LNetError: 46080:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1024.345950] LNet: Removed LNI 192.168.201.55@tcp [ 1024.366372] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1024.452121] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1024.485777] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1025.106837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1025.138233] LNet: Removed LNI 192.168.201.55@tcp [ 1025.188243] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1025.338394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1025.378903] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1026.059369] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1026.178066] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1026.187623] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1026.192393] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1028.787665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1028.821939] LNet: 46188:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1028.831611] LNet: 46188:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 1028.841162] LNetError: 46188:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1028.852877] LNetError: 46188:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1028.871772] LNet: Removed LNI 192.168.201.55@tcp [ 1028.896936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1028.995623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 1029.037695] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1037.673960] Lustre: DEBUG MARKER: == sanity-lnet test 153: Check import of ip2nets with IP address ranges ========================================================== 09:24:01 (1789133041) [ 1037.834652] LNet: 46778:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1037.843635] LNetError: 46778:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1037.863056] LNet: Removed LNI 192.168.201.55@tcp [ 1039.005861] Key type .llcrypt unregistered [ 1039.009084] Key type ._llcrypt unregistered [ 1046.127136] Key type ._llcrypt registered [ 1046.133381] Key type .llcrypt registered [ 1046.527844] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1046.579582] alg: No test for adler32 (adler32-zlib) [ 1047.444209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1047.511913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1047.536544] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1048.449825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1048.486597] LNet: 47189:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1048.501895] LNetError: 47189:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1048.541598] LNet: Removed LNI 192.168.201.55@tcp [ 1048.559147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1048.634418] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-153.yaml [ 1048.673944] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1055.743801] Lustre: DEBUG MARKER: == sanity-lnet test 154: Check import of ip2nets with stepped IP address ranges ========================================================== 09:24:20 (1789133060) [ 1055.889237] LNet: 47780:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1055.893623] LNetError: 47780:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1055.908147] LNet: Removed LNI 192.168.201.55@tcp [ 1056.542671] Key type .llcrypt unregistered [ 1056.544436] Key type ._llcrypt unregistered [ 1066.127436] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1066.631281] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1066.651177] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1066.665280] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1068.507558] Key type ._llcrypt registered [ 1068.510465] Key type .llcrypt registered [ 1068.984895] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1069.025213] alg: No test for adler32 (adler32-zlib) [ 1070.051226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1070.135658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1070.177937] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1071.508763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1071.549364] LNet: 48227:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1071.563277] LNetError: 48227:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1071.607679] LNet: Removed LNI 192.168.201.55@tcp [ 1071.647487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1071.718810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-154.yaml [ 1071.738815] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1079.698951] Lustre: DEBUG MARKER: == sanity-lnet test 155: Check ip2nets import failure with no matching rules ========================================================== 09:24:44 (1789133084) [ 1079.954253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1079.999733] LNet: 48828:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1080.006708] LNetError: 48828:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1080.022177] LNet: Removed LNI 192.168.201.55@tcp [ 1080.051580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1080.194462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-155.yaml [ 1087.212433] Lustre: DEBUG MARKER: == sanity-lnet test 156: Check import of ip2nets with comma-separated IP ranges ========================================================== 09:24:52 (1789133092) [ 1087.669644] LNet: 49411:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1088.501823] Key type .llcrypt unregistered [ 1088.507096] Key type ._llcrypt unregistered [ 1095.433577] Key type ._llcrypt registered [ 1095.436854] Key type .llcrypt registered [ 1095.673373] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1095.689097] alg: No test for adler32 (adler32-zlib) [ 1096.714287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1096.839167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1096.884909] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1098.251652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1098.287581] LNet: 49812:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1098.293958] LNetError: 49812:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1098.307723] LNet: Removed LNI 192.168.201.55@tcp [ 1098.335829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1098.452435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-156.yaml [ 1098.512324] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1106.377695] Lustre: DEBUG MARKER: == sanity-lnet test 157: Check import failure with malformed ip2nets YAML ========================================================== 09:25:11 (1789133111) [ 1106.682967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1106.711823] LNet: 50406:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1106.717481] LNetError: 50406:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1106.743070] LNet: Removed LNI 192.168.201.55@tcp [ 1106.811994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1106.923906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-157.yaml [ 1114.342472] Lustre: DEBUG MARKER: == sanity-lnet test 158: Check import of ip2nets with single IPv6 address ========================================================== 09:25:18 (1789133118) [ 1115.920393] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_158 Interface has no IPv6 [ 1118.256666] Lustre: DEBUG MARKER: == sanity-lnet test 159: Check import of ip2nets with IPv6 /128 CIDR notation ========================================================== 09:25:22 (1789133122) [ 1120.258331] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_159 Interface has no IPv6 [ 1122.385894] Lustre: DEBUG MARKER: == sanity-lnet test 160: Check correct application of multiple IPv6 ip2nets rules ========================================================== 09:25:27 (1789133127) [ 1124.305699] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_160 Interface has no IPv6 [ 1126.055219] Lustre: DEBUG MARKER: == sanity-lnet test 161: Check import of ip2nets with various IPv6 prefix lengths ========================================================== 09:25:31 (1789133131) [ 1128.444547] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_161 Interface has no IPv6 [ 1130.911328] Lustre: DEBUG MARKER: == sanity-lnet test 162: Check ip2nets import failure with non-matching IPv6 ========================================================== 09:25:35 (1789133135) [ 1133.768246] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_162 Interface has no IPv6 [ 1136.465946] Lustre: DEBUG MARKER: == sanity-lnet test 170: Check CPTs and tunables parsing when importing ========================================================== 09:25:40 (1789133140) [ 1136.818833] LNet: 52460:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1137.705052] Key type .llcrypt unregistered [ 1137.712327] Key type ._llcrypt unregistered [ 1147.768776] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1147.914168] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1147.919130] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1147.921951] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1150.666234] Key type ._llcrypt registered [ 1150.673904] Key type .llcrypt registered [ 1151.043190] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1151.067481] alg: No test for adler32 (adler32-zlib) [ 1152.212797] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1152.277765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1152.303228] LNet: Added LNI 192.168.201.55@tcp [8/512/0/180] [ 1152.310513] LNet: Added LNI 10.1.2.3@tcp [8/512/0/180] [ 1152.313932] LNet: 52786:0:(api-ni.c:2986:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1152.445844] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1152.474840] LNet: 52805:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1152.487683] LNetError: 52805:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1152.497703] LNet: Removed LNI 10.1.2.3@tcp [ 1152.508858] LNetError: 52805:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1152.541682] LNet: Removed LNI 192.168.201.55@tcp [ 1152.561772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1152.623430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1152.656852] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1152.751262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1152.786375] LNet: 52829:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1152.794422] LNetError: 52829:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1152.827834] LNet: Removed LNI 192.168.201.55@tcp [ 1152.856597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1153.009204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1153.040974] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1153.051584] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1153.054598] LNet: 52837:0:(api-ni.c:2986:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1153.154757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1153.184548] LNet: 52855:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1153.190971] LNetError: 52855:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1153.198869] LNet: Removed LNI 10.1.2.3@tcp [ 1153.201410] LNetError: 52855:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1153.219416] LNet: Removed LNI 192.168.201.55@tcp [ 1153.261817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1153.338479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1153.396643] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1153.402379] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1153.404222] LNet: 52863:0:(api-ni.c:2986:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1153.773718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1153.795821] LNet: 52889:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1153.802304] LNetError: 52889:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1153.805157] LNet: Removed LNI 10.1.2.3@tcp [ 1153.808252] LNetError: 52889:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1153.822643] LNet: Removed LNI 192.168.201.55@tcp [ 1153.846532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1153.929858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1153.972861] LNet: Added LNI 192.168.201.55@tcp [8/256/0/182] [ 1154.005876] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 1154.109503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1154.126853] LNet: 52917:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1154.135608] LNetError: 52917:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1154.146926] LNet: Removed LNI 10.1.2.3@tcp [ 1154.153124] LNetError: 52917:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1154.188186] LNet: Removed LNI 192.168.201.55@tcp [ 1154.234687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1154.366200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 1154.413661] LNet: Added LNI 192.168.201.55@tcp [8/256/0/182] [ 1154.430889] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 1154.432868] LNet: 52925:0:(api-ni.c:2986:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 1154.629930] LNet: Removed LNI 10.1.2.3@tcp [ 1154.665531] LNet: Removed LNI 192.168.201.55@tcp [ 1155.592883] Key type .llcrypt unregistered [ 1155.595207] Key type ._llcrypt unregistered [ 1172.899316] Key type ._llcrypt registered [ 1172.901858] Key type .llcrypt registered [ 1174.336955] Lustre: DEBUG MARKER: == sanity-lnet test 180: FastReg map error should not leak descriptors ========================================================== 09:26:19 (1789133179) [ 1176.695667] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_180 Need o2ib NETTYPE [ 1179.103717] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 09:26:23 (1789133183) [ 1179.953382] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1179.988740] alg: No test for adler32 (adler32-zlib) [ 1180.974893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1181.029146] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1181.067851] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1182.162681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --ip2net tcp 192.168.201.55 [ 1182.191439] LNet: 54196:0:(config.c:1738:lnet_parse_ip2nets()) Kernel parsing of ip2nets is deprecated. Consider converting to LNet YAML configuration. [ 1182.200886] LNet: Removed LNI 192.168.201.55@tcp [ 1190.601413] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 09:26:35 (1789133195) [ 1190.869377] LNet: 54766:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1190.874218] LNetError: 54766:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1190.888146] LNet: Removed LNI 192.168.201.55@tcp [ 1191.546412] Key type .llcrypt unregistered [ 1191.551496] Key type ._llcrypt unregistered [ 1200.391201] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1200.779710] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1203.776441] Key type ._llcrypt registered [ 1203.778703] Key type .llcrypt registered [ 1204.138644] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1204.156781] alg: No test for adler32 (adler32-zlib) [ 1205.891861] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1214.998424] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 09:26:59 (1789133219) [ 1215.529248] LNet: 55685:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1215.551852] LNetError: 55685:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1215.579301] LNet: Removed LNI 10.1.2.3@tcp [ 1216.707129] Key type .llcrypt unregistered [ 1216.709873] Key type ._llcrypt unregistered [ 1225.756780] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1225.913347] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1225.915742] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1225.917835] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1228.906075] Key type ._llcrypt registered [ 1228.918245] Key type .llcrypt registered [ 1229.321840] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1229.386251] alg: No test for adler32 (adler32-zlib) [ 1230.373672] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1236.796598] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 09:27:21 (1789133241) [ 1238.170347] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_202 kernel ip2nets deprecated in 2.17.53 [ 1239.954090] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 09:27:24 (1789133244) [ 1240.315874] LNet: 56876:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1240.318841] LNetError: 56876:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1240.333990] LNet: Removed LNI 10.1.2.3@tcp [ 1241.122728] Key type .llcrypt unregistered [ 1241.125379] Key type ._llcrypt unregistered [ 1246.923655] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1247.180826] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1247.186918] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1247.196697] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1250.032380] Key type ._llcrypt registered [ 1250.034708] Key type .llcrypt registered [ 1250.322871] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1250.336491] alg: No test for adler32 (adler32-zlib) [ 1251.298797] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1251.385408] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1263.426461] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 09:27:47 (1789133267) [ 1264.210400] LNet: 58111:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1264.216368] LNetError: 58111:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1264.240525] LNet: Removed LNI 10.1.2.3@tcp [ 1264.968562] Key type .llcrypt unregistered [ 1264.972648] Key type ._llcrypt unregistered [ 1272.325920] Key type ._llcrypt registered [ 1272.337766] Key type .llcrypt registered [ 1272.682247] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1272.691312] alg: No test for adler32 (adler32-zlib) [ 1273.638777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1273.670165] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1282.943438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 1283.885482] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_interrupt -r 1 [ 1283.912665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_interrupt -r 1 [ 1283.942501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1283.963422] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1284.368579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1284.446579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1284.515216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1284.567582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1284.621766] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1284.684752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1284.742961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1284.798804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1284.817303] LNet: 58554:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1284.837647] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1284.877793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1284.945478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1284.961753] LNet: 1 local NIs in recovery (showing 1): 192.168.201.55@tcp [ 1284.969587] LNet: There was an unexpected network error while writing to 192.168.201.55: rc = -22 [ 1285.003279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1285.060285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1285.100496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1285.139811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1285.756828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_dropped -r 1 [ 1285.809084] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_dropped -r 1 [ 1285.869626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1286.154374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1286.180815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1286.204316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1286.227705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1286.251402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1286.282818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1286.326036] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1286.368282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1286.385659] LNet: 58663:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1286.399701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1286.432318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1286.468685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1286.501398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1286.529638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1286.552162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1286.577847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1287.341564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_aborted -r 1 [ 1287.407465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_aborted -r 1 [ 1287.472508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1287.490104] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1287.498517] LNet: Skipped 1 previous similar message [ 1287.793835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1287.826346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1287.851819] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1287.900095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1287.947682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1287.985902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1288.025899] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1288.066670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1288.086613] LNet: 58772:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1288.100780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1288.150402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1288.194361] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1288.245655] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1288.290995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1288.367455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1288.420853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1289.048393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_no_route -r 1 [ 1289.122057] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_no_route -r 1 [ 1289.150336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1289.424438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1289.457757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1289.494548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1289.530212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1289.571420] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1289.615774] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1289.643863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1289.670558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1289.693710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1289.727904] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1289.759955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1289.782714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1289.807990] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1289.830679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1289.848184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1290.335231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_timeout -r 1 [ 1290.367890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_timeout -r 1 [ 1290.417642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1290.496214] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1290.521262] LNet: Skipped 1 previous similar message [ 1290.841183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1290.895367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1290.935344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1290.973525] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1291.004822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1291.024391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1291.056228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1291.092208] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1291.106575] LNet: 58991:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1291.110723] LNet: 58991:0:(api-ni.c:296:recovery_interval_set()) Skipped 1 previous similar message [ 1291.125321] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1291.194587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1291.222736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1291.252715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1291.313829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1291.358691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1291.405296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1291.916143] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_error -r 1 [ 1291.962569] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_error -r 1 [ 1291.989476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1292.224656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1292.251605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1292.274629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1292.299705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1292.323982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1292.346675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1292.367380] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1292.387843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1292.410065] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1292.432678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1292.456499] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1292.478320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1292.503845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1292.524292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1292.552518] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1292.787829] LNet: 59145:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1292.795603] LNetError: 59145:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1292.814839] LNet: Removed LNI 192.168.201.55@tcp [ 1293.270495] Key type .llcrypt unregistered [ 1293.272075] Key type ._llcrypt unregistered [ 1310.998111] Key type ._llcrypt registered [ 1310.999567] Key type .llcrypt registered [ 1312.155531] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 09:28:37 (1789133317) [ 1313.088465] Key type .llcrypt unregistered [ 1313.090252] Key type ._llcrypt unregistered [ 1319.892407] Key type ._llcrypt registered [ 1319.895085] Key type .llcrypt registered [ 1320.143547] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1320.165726] alg: No test for adler32 (adler32-zlib) [ 1321.097776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1321.143332] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1328.893592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 1338.612957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1338.670162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1338.697163] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 1340.198817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_interrupt -r 1 [ 1340.242103] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e local_interrupt -r 1 [ 1340.279662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_interrupt -r 1 [ 1340.309887] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e local_interrupt -r 1 [ 1340.342674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e local_interrupt -r 1 [ 1340.379110] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e local_interrupt -r 1 [ 1340.414067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e local_interrupt -r 1 [ 1340.440938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e local_interrupt -r 1 [ 1340.466866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1340.485713] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1340.488809] LNet: 1 local NIs in recovery (showing 1): 192.168.201.55@tcp [ 1340.725770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1340.750635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1340.794648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1340.835641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1340.878296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1340.924433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1340.971670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1341.019631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1341.068231] LNet: 60626:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1341.097662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1341.184559] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1341.281516] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1341.371915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1341.442645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1341.487984] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1341.533263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1342.287246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_dropped -r 1 [ 1342.371124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e local_dropped -r 1 [ 1342.421467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_dropped -r 1 [ 1342.463462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e local_dropped -r 1 [ 1342.538737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e local_dropped -r 1 [ 1342.591523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e local_dropped -r 1 [ 1342.644425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e local_dropped -r 1 [ 1342.691087] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e local_dropped -r 1 [ 1342.729179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1342.755325] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1342.760932] LNet: Skipped 2 previous similar messages [ 1343.073463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1343.115065] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1343.158224] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1343.183460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1343.217133] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1343.255733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1343.277622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1343.306653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1343.329717] LNet: 60747:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1343.351114] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1343.385776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1343.413042] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1343.453461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1343.488833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1343.517793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1343.548981] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1344.078180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_aborted -r 1 [ 1344.126809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e local_aborted -r 1 [ 1344.170032] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_aborted -r 1 [ 1344.225435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e local_aborted -r 1 [ 1344.281390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e local_aborted -r 1 [ 1344.354823] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e local_aborted -r 1 [ 1344.423656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e local_aborted -r 1 [ 1344.489422] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e local_aborted -r 1 [ 1344.529740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1344.554268] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1344.557407] LNet: Skipped 2 previous similar messages [ 1344.790102] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1344.818340] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1344.864890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1344.894489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1344.920638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1344.948553] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1344.992233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1345.017604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1345.033734] LNet: 60869:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1345.053482] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1345.100595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1345.157073] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1345.188145] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1345.214616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1345.251411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1345.322313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1345.966661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_no_route -r 1 [ 1346.002995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e local_no_route -r 1 [ 1346.045588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_no_route -r 1 [ 1346.077736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e local_no_route -r 1 [ 1346.114688] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e local_no_route -r 1 [ 1346.172308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e local_no_route -r 1 [ 1346.216735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e local_no_route -r 1 [ 1346.268872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e local_no_route -r 1 [ 1346.334906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1346.646378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1346.679698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1346.704703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1346.743466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1346.774553] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1346.812725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1346.867344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1346.911418] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1346.939019] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1346.973712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1347.006958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1347.041929] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1347.069151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1347.103582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1347.142514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1347.644219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_timeout -r 1 [ 1347.678435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e local_timeout -r 1 [ 1347.716176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_timeout -r 1 [ 1347.768623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e local_timeout -r 1 [ 1347.828432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e local_timeout -r 1 [ 1347.857254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e local_timeout -r 1 [ 1347.893634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e local_timeout -r 1 [ 1347.920368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e local_timeout -r 1 [ 1347.955575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1347.972973] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1347.982129] LNet: Skipped 5 previous similar messages [ 1348.179539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1348.210663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1348.242557] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1348.272319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1348.297385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1348.334780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1348.379640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1348.422969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1348.445362] LNet: 61111:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1348.454528] LNet: 61111:0:(api-ni.c:296:recovery_interval_set()) Skipped 1 previous similar message [ 1348.473413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1348.505669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1348.531728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1348.553364] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1348.578942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1348.605289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1348.632736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1349.067268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_error -r 1 [ 1349.115087] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e local_error -r 1 [ 1349.142687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_error -r 1 [ 1349.179754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e local_error -r 1 [ 1349.219558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e local_error -r 1 [ 1349.252812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e local_error -r 1 [ 1349.291809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e local_error -r 1 [ 1349.344894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e local_error -r 1 [ 1349.396970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1349.628213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1349.662366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1349.703168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1349.741822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1349.810226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1349.848407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1349.896621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1349.946500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1350.044306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1350.083124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1350.129404] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1350.187571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1350.237320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1350.270186] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1350.300388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1355.624473] LNet: 61313:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1355.637771] LNetError: 61313:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1355.645366] LNet: Removed LNI 192.168.201.55@tcp [ 1356.719669] LNet: Removed LNI 192.168.201.55@tcp1 [ 1357.235691] Key type .llcrypt unregistered [ 1357.240190] Key type ._llcrypt unregistered [ 1373.593785] Key type ._llcrypt registered [ 1373.598603] Key type .llcrypt registered [ 1375.009113] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 09:29:39 (1789133379) [ 1375.776393] Key type .llcrypt unregistered [ 1375.779818] Key type ._llcrypt unregistered [ 1381.920981] Key type ._llcrypt registered [ 1381.924534] Key type .llcrypt registered [ 1382.097303] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1382.107649] alg: No test for adler32 (adler32-zlib) [ 1383.032695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1383.086187] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1391.268272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 1392.250730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e remote_dropped -r 1 [ 1392.334227] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e remote_dropped -r 1 [ 1392.403230] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1392.447184] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1392.721580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1392.760741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1392.815967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1392.880506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1392.948461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1393.037050] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1393.082884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1393.114975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1393.133767] LNet: 62567:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1393.150951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1393.174807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1393.200372] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1393.224683] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1393.262343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1393.309927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1393.384560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1393.982287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e remote_error -r 1 [ 1394.031411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e remote_error -r 1 [ 1394.075842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1394.097158] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1394.422923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1394.467825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1394.496444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1394.544560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1394.611703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1394.653790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1394.686727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1394.731395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1394.758165] LNet: 62676:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1394.796178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1394.834708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1394.880911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1394.930338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1394.995180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1395.036691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1395.061968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1395.644329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e remote_timeout -r 1 [ 1395.682320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e remote_timeout -r 1 [ 1395.721659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1395.740772] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1395.947546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1395.972608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1395.993509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1396.033092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1396.077814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1396.109211] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1396.144828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1396.180500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1396.207846] LNet: 62785:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1396.231549] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1396.287528] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1396.323607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1396.399385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1396.469875] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1396.538108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1396.571862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1396.973805] LNet: 62831:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1396.984934] LNetError: 62831:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1398.056084] LNet: Removed LNI 192.168.201.55@tcp [ 1398.657599] Key type .llcrypt unregistered [ 1398.659403] Key type ._llcrypt unregistered [ 1417.544110] Key type ._llcrypt registered [ 1417.545711] Key type .llcrypt registered [ 1418.633597] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 09:30:23 (1789133423) [ 1419.619556] Key type .llcrypt unregistered [ 1419.621669] Key type ._llcrypt unregistered [ 1426.929959] Key type ._llcrypt registered [ 1426.936152] Key type .llcrypt registered [ 1427.165044] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1427.202312] alg: No test for adler32 (adler32-zlib) [ 1428.227992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1428.258617] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1436.060888] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 1446.876627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1446.937740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1446.966899] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 1448.159612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e remote_dropped -r 1 [ 1448.190549] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e remote_dropped -r 1 [ 1448.214396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e remote_dropped -r 1 [ 1448.230900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e remote_dropped -r 1 [ 1448.247961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e remote_dropped -r 1 [ 1448.265622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e remote_dropped -r 1 [ 1448.283222] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e remote_dropped -r 1 [ 1448.304616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e remote_dropped -r 1 [ 1448.331500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1448.349898] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1448.358681] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.155@tcp [ 1448.553082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1448.592848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1448.636882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1448.678952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1448.739715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1448.790534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1448.824963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1448.863836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1448.887469] LNet: 64305:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1448.913545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1448.947773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1448.989932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1449.035685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1449.136446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1449.218302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1449.254772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1449.290850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1449.885615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e remote_error -r 1 [ 1449.921099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e remote_error -r 1 [ 1449.955194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e remote_error -r 1 [ 1450.003652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e remote_error -r 1 [ 1450.058356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e remote_error -r 1 [ 1450.081385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e remote_error -r 1 [ 1450.104601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e remote_error -r 1 [ 1450.124413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e remote_error -r 1 [ 1450.146791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1450.161206] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1450.164899] LNet: Skipped 2 previous similar messages [ 1450.408953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1450.449311] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1450.520502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1450.560532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1450.608381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1450.646595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1450.683700] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1450.725185] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1450.746903] LNet: 64428:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1450.769040] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1450.819517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1450.853124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1450.878693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1450.915571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1450.955839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1451.032656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1451.092702] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1451.613789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e remote_timeout -r 1 [ 1451.676108] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e remote_timeout -r 1 [ 1451.727223] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e remote_timeout -r 1 [ 1451.771218] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e remote_timeout -r 1 [ 1451.802906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e remote_timeout -r 1 [ 1451.839580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e remote_timeout -r 1 [ 1451.880662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e remote_timeout -r 1 [ 1451.919189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e remote_timeout -r 1 [ 1451.971558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.155@tcp [ 1451.990727] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1452.315856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1452.380467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1452.420504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1452.459877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1452.487374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1452.519533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1452.571658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1452.607346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1452.623538] LNet: 64551:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1452.642332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1452.679200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1452.707847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1452.742720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1452.779365] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1452.819117] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1452.844596] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1452.909836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1458.230627] LNet: 64633:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1458.235790] LNetError: 64633:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1458.241773] LNet: Removed LNI 192.168.201.55@tcp [ 1458.253194] LNet: Removed LNI 192.168.201.55@tcp1 [ 1458.754875] Key type .llcrypt unregistered [ 1458.758782] Key type ._llcrypt unregistered [ 1480.892611] Key type ._llcrypt registered [ 1480.894739] Key type .llcrypt registered [ 1482.606933] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 09:31:27 (1789133487) [ 1484.213888] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_208 kernel ip2nets deprecated in 2.17.53 [ 1486.098708] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 09:31:31 (1789133491) [ 1487.090511] Key type .llcrypt unregistered [ 1487.095281] Key type ._llcrypt unregistered [ 1493.914867] Key type ._llcrypt registered [ 1493.920722] Key type .llcrypt registered [ 1494.084182] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1494.097976] alg: No test for adler32 (adler32-zlib) [ 1495.101242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1495.157399] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1504.064184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 1504.874419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e network_timeout -r 1 [ 1504.918328] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e network_timeout -r 1 [ 1504.953970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 1504.973572] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1505.249506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1505.282627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1505.314707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1505.342365] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1505.377602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1505.413329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1505.441492] LNet: 1 local NIs in recovery (showing 1): 192.168.201.55@tcp [ 1505.464825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1505.520897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1505.549507] LNet: 66184:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1505.567952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1505.642804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1505.710746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1505.745941] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1505.785497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1505.836710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1506.144440] LNet: 66227:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1506.152744] LNetError: 66227:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1507.175754] LNet: Removed LNI 192.168.201.55@tcp [ 1507.702642] Key type .llcrypt unregistered [ 1507.705293] Key type ._llcrypt unregistered [ 1526.093519] Key type ._llcrypt registered [ 1526.099732] Key type .llcrypt registered [ 1526.506322] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1526.555123] alg: No test for adler32 (adler32-zlib) [ 1527.527925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1527.562350] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1535.760189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 1545.047611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1545.086837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1545.107670] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 1546.676903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e network_timeout -r 1 [ 1546.709339] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e network_timeout -r 1 [ 1546.746913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e network_timeout -r 1 [ 1546.793467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e network_timeout -r 1 [ 1546.842689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e network_timeout -r 1 [ 1546.882797] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e network_timeout -r 1 [ 1546.931490] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e network_timeout -r 1 [ 1546.985301] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e network_timeout -r 1 [ 1547.037679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 1547.059722] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1547.308865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1547.338425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 1547.390699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1547.436947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1547.490724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1547.556955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1547.597672] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1547.633441] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1547.654680] LNet: 67112:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1547.695679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1547.738684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1547.761624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1547.799196] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1547.831809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1547.866100] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1552.717029] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1553.079840] LNet: 67190:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1553.084917] LNetError: 67190:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1553.091226] LNet: Removed LNI 192.168.201.55@tcp [ 1553.104254] LNet: Removed LNI 192.168.201.55@tcp1 [ 1553.745249] Key type .llcrypt unregistered [ 1553.749922] Key type ._llcrypt unregistered [ 1575.171193] Key type ._llcrypt registered [ 1575.172840] Key type .llcrypt registered [ 1577.432729] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 09:33:01 (1789133581) [ 1578.377335] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1578.417392] alg: No test for adler32 (adler32-zlib) [ 1579.604191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1579.741937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1579.782039] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1581.111563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1581.149607] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 1583.174796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 1583.319936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 1583.400180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 1583.490842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 1583.544095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1583.575602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1583.686795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.55@tcp [ 1583.746419] LNetError: 68293:0:(api-ni.c:4473:lnet_ni_set_healthv()) manually adding local NI 192.168.201.55@tcp to recovery [ 1583.793504] LNet: 1 local NIs in recovery (showing 1): 192.168.201.55@tcp [ 1583.796171] LNet: There was an unexpected network error while writing to 192.168.201.55: rc = -22 [ 1585.837789] LNet: There was an unexpected network error while writing to 192.168.201.55: rc = -22 [ 1586.935294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1589.927289] LNet: There was an unexpected network error while writing to 192.168.201.55: rc = -22 [ 1591.065242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1591.129629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1591.256283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1591.269831] LNet: 68390:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1591.273840] LNetError: 68390:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1592.291178] LNet: Removed LNI 192.168.201.55@tcp [ 1592.305456] LNetError: 68390:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1592.338433] LNet: Removed LNI 192.168.201.55@tcp1 [ 1592.385320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1592.431601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1592.445952] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1593.119865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1593.152972] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 1594.347531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 1594.392764] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1594.450552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 1594.516911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 1594.571901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 1594.606682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1594.650946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1594.682234] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.55@tcp [ 1594.701741] LNetError: 68662:0:(api-ni.c:4473:lnet_ni_set_healthv()) manually adding local NI 192.168.201.55@tcp to recovery [ 1595.490720] LNet: There was an unexpected network error while writing to 192.168.201.55: rc = -22 [ 1598.574453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1601.632509] LNet: There was an unexpected network error while writing to 192.168.201.55: rc = -22 [ 1601.659265] LNet: Skipped 1 previous similar message [ 1606.136548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1606.229951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1606.290958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1619.139008] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 09:33:42 (1789133622) [ 1619.612395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1619.686450] LNet: 69373:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1619.701212] LNetError: 69373:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1619.715834] LNet: Removed LNI 192.168.201.55@tcp [ 1619.741400] LNet: Removed LNI 192.168.201.55@tcp1 [ 1619.859188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1620.006417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1620.115756] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1621.957391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1622.045679] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 1623.284908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 1623.347390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 1623.375818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 1623.417662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 1623.471936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1623.517717] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1623.555540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.55@tcp --health 0 [ 1623.680188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1624.033788] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.55@tcp [ 1624.035346] LNet: There was an unexpected network error while writing to 192.168.201.55: rc = -22 [ 1624.053500] LNet: Skipped 1 previous similar message [ 1638.948495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1639.239609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1639.318135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1639.386782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.55@tcp --health 500 [ 1639.646870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1645.641479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1645.721303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1645.779567] LNet: 69850:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1645.788533] LNetError: 69850:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1645.796708] LNetError: 69850:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1645.805359] LNet: Removed LNI 192.168.201.55@tcp [ 1645.817153] LNet: Removed LNI 192.168.201.55@tcp1 [ 1645.862545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1645.967955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1646.023812] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1647.219835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1647.272647] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 1648.705606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 1648.758709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1648.839854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 1648.888966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 1648.952305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 1649.027215] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 1649.108272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 1649.192364] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.201.55@tcp --health 0 [ 1650.017192] LNet: There was an unexpected network error while writing to 192.168.201.55: rc = -22 [ 1650.020609] LNet: Skipped 3 previous similar messages [ 1650.831496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1660.814646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1660.844666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1660.889737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1668.695385] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 09:34:33 (1789133673) [ 1669.430979] LNet: 70886:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1669.453220] LNetError: 70886:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1669.463026] LNetError: 70886:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1670.497234] LNet: Removed LNI 192.168.201.55@tcp [ 1670.506686] LNet: Removed LNI 192.168.201.55@tcp1 [ 1671.402517] Key type .llcrypt unregistered [ 1671.406218] Key type ._llcrypt unregistered [ 1679.219506] Key type ._llcrypt registered [ 1679.229851] Key type .llcrypt registered [ 1679.642212] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1679.681618] alg: No test for adler32 (adler32-zlib) [ 1680.724570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1680.759414] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1690.267905] Lustre: DEBUG MARKER: Initial discovery [ 1690.426726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.155@tcp [ 1692.871547] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag [ 1692.911735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 -e local_error [ 1692.966070] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.155@tcp [ 1692.985225] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1693.005588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1693.044279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.201.155@tcp -r 1 -m GET -l 3 [ 1693.090997] LNet: 1 local NIs in recovery (showing 1): 192.168.201.55@tcp [ 1693.131240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.201.155@tcp -r 1 -m GET -e local_error [ 1693.185221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.201.155@tcp -r 1 -m PUT -l 6 [ 1694.908350] Lustre: DEBUG MARKER: Force oleg155-client.virtnet to discover 192.168.201.155@tcp (in background) [ 1695.010844] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.201.155@tcp [ 1696.361436] Lustre: DEBUG MARKER: Force oleg155-server to discover 192.168.201.55@tcp [ 1698.080484] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1698.821270] Lustre: DEBUG MARKER: Wait for 71644 [ 1700.665706] Lustre: DEBUG MARKER: Finished wait on 71644 [ 1700.888761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay del -a [ 1700.947130] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 1700.954410] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 1701.464038] LNet: 72106:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1701.475987] LNetError: 72106:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1701.506812] LNet: Removed LNI 192.168.201.55@tcp [ 1702.086590] Key type .llcrypt unregistered [ 1702.090219] Key type ._llcrypt unregistered [ 1720.953761] Key type ._llcrypt registered [ 1720.957961] Key type .llcrypt registered [ 1722.736356] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 09:35:27 (1789133727) [ 1723.937555] Key type .llcrypt unregistered [ 1723.939674] Key type ._llcrypt unregistered [ 1732.245364] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1732.601969] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1732.607887] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1732.613576] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1735.477812] Key type ._llcrypt registered [ 1735.483991] Key type .llcrypt registered [ 1735.744418] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1735.772797] alg: No test for adler32 (adler32-zlib) [ 1736.779712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1736.862898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1736.912250] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1737.688861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1737.725388] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1749.461045] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 09:35:53 (1789133753) [ 1749.686638] LNet: 74193:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1749.693485] LNetError: 74193:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1749.699433] LNet: Removed LNI 10.1.2.3@tcp [ 1749.709671] LNet: Removed LNI 192.168.201.55@tcp [ 1750.397499] Key type .llcrypt unregistered [ 1750.399522] Key type ._llcrypt unregistered [ 1758.280758] Key type ._llcrypt registered [ 1758.283621] Key type .llcrypt registered [ 1758.498201] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1758.520036] alg: No test for adler32 (adler32-zlib) [ 1759.414613] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1759.481857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1759.508707] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1760.301468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1760.322514] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1761.316075] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 10.1.2.3@tcp 192.168.201.55@tcp [ 1762.400395] LNetError: Unexpected error -22 connecting to 192.168.201.55@tcp at host 192.168.201.55:988 [ 1768.250379] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 09:36:13 (1789133773) [ 1768.454721] LNet: 75330:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1768.459519] LNetError: 75330:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 1768.467921] LNet: Removed LNI 10.1.2.3@tcp [ 1768.476986] LNet: Removed LNI 192.168.201.55@tcp [ 1769.312484] Key type .llcrypt unregistered [ 1769.315070] Key type ._llcrypt unregistered [ 1776.267874] Key type ._llcrypt registered [ 1776.269940] Key type .llcrypt registered [ 1776.534539] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1776.554811] alg: No test for adler32 (adler32-zlib) [ 1777.531159] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1777.617379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1777.656346] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 1778.824140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1778.845299] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 1779.895499] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 192.168.201.55@tcp1 --nid 192.168.201.55@tcp2 [ 1785.238259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.55@tcp1 192.168.201.55@tcp2 [ 1785.280801] LNetError: 76170:0:(lib-move.c:2101:lnet_handle_find_routed_path()) Unable to send message from 192.168.201.55@tcp1 to 192.168.201.55@tcp2 - Route table may be misconfigured [ 1785.307577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.55@tcp2 192.168.201.55@tcp1 [ 1794.195717] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 09:36:39 (1789133799) [ 1794.281586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1794.314188] LNet: 76738:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1794.328051] LNetError: 76738:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1795.360201] LNet: Removed LNI 192.168.201.55@tcp1 [ 1795.364432] LNetError: 76738:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1795.389222] LNet: Removed LNI 192.168.201.55@tcp2 [ 1795.453597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1795.556618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1795.606056] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1796.368858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1796.387977] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 1797.152745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 1797.186248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.55@tcp 192.168.201.55@tcp [ 1797.219669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.55@tcp1 192.168.201.55@tcp1 [ 1797.265900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.55@tcp -d 192.168.201.55@tcp -e network_timeout [ 1797.325493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -e network_timeout [ 1797.359841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -e network_timeout [ 1797.396261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -e network_timeout [ 1797.432481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1797.451717] LNet: There was an unexpected network error while writing to 192.168.201.55: rc = -22 [ 1797.472107] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 1797.539454] LNet: 1 local NIs in recovery (showing 1): 192.168.201.55@tcp [ 1797.566549] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 1797.673246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 1804.155374] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 09:36:49 (1789133809) [ 1804.262588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1804.300523] LNet: 77599:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1804.311153] LNetError: 77599:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1805.344282] LNet: Removed LNI 192.168.201.55@tcp [ 1806.380415] LNet: Removed LNI 192.168.201.55@tcp1 [ 1806.415597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1806.509303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 0@lo [ 1806.864462] LNet: 77648:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1807.283976] Key type .llcrypt unregistered [ 1807.286329] Key type ._llcrypt unregistered [ 1822.902766] Key type ._llcrypt registered [ 1822.907171] Key type .llcrypt registered [ 1824.700231] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 09:37:08 (1789133828) [ 1825.138542] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1825.170637] alg: No test for adler32 (adler32-zlib) [ 1826.140663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1827.572528] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces [ 1829.659302] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 09:37:14 (1789133834) [ 1829.795669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1829.861101] LNet: 78741:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1829.912818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1830.040344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1830.080091] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1831.002590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1831.033946] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 1832.216589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1832.303081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp1 [ 1832.340641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp1 [ 1839.243392] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 09:37:24 (1789133844) [ 1841.033814] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_220 Need at least 2 remote nodes found oleg155-server [ 1842.950232] Lustre: DEBUG MARKER: == sanity-lnet test 221: Add routes w/DD disabled - check aliveness ========================================================== 09:37:27 (1789133847) [ 1844.980574] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_221 Need at least 2 remote nodes found oleg155-server [ 1847.535522] Lustre: DEBUG MARKER: == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 09:37:32 (1789133852) [ 1849.409974] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_222 Need at least 2 remote nodes found oleg155-server [ 1851.021808] Lustre: DEBUG MARKER: == sanity-lnet test 223: Check avoid_asym_router_failure=1 w/DD disabled ========================================================== 09:37:36 (1789133856) [ 1852.800735] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_223 Need at least 2 remote nodes found oleg155-server [ 1855.211825] Lustre: DEBUG MARKER: == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 09:37:39 (1789133859) [ 1857.259871] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_224 Need at least 2 remote nodes found oleg155-server [ 1860.290487] Lustre: DEBUG MARKER: == sanity-lnet test 225: Check avoid_asym_router_failure=0 w/DD disabled ========================================================== 09:37:44 (1789133864) [ 1863.122476] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_225 Need at least 2 remote nodes found oleg155-server [ 1865.596591] Lustre: DEBUG MARKER: == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 09:37:50 (1789133870) [ 1869.437158] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_226 Need at least 3 remote nodes found oleg155-server [ 1872.192802] Lustre: DEBUG MARKER: == sanity-lnet test 227: Check router peer health w/DD disabled ========================================================== 09:37:56 (1789133876) [ 1875.592577] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_227 Need at least 3 remote nodes found oleg155-server [ 1878.611669] Lustre: DEBUG MARKER: == sanity-lnet test 228: Routes should stay up when health is decremented ========================================================== 09:38:02 (1789133882) [ 1882.512709] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_228 Need at least 3 remote nodes found oleg155-server [ 1884.946105] Lustre: DEBUG MARKER: == sanity-lnet test 230: Test setting conns-per-peer ===== 09:38:09 (1789133889) [ 1885.042595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1885.093249] LNet: 82621:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1885.110670] LNetError: 82621:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1885.117634] LNet: Removed LNI 192.168.201.55@tcp [ 1886.192175] LNet: Removed LNI 192.168.201.55@tcp1 [ 1886.279730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1886.416464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1886.447970] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1887.739661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 4 [ 1887.950946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1888.204749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1888.232320] LNet: 82776:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1888.240610] LNetError: 82776:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1888.249849] LNetError: 82776:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1889.353379] LNet: Removed LNI 192.168.201.55@tcp [ 1889.471821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1889.536082] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 5 [ 1889.581868] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1890.608480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1890.947642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1891.004539] LNet: 82922:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1891.015495] LNetError: 82922:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1892.081250] LNet: Removed LNI 192.168.201.55@tcp [ 1892.121282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1892.271839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1892.343057] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1894.224945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 6 [ 1894.407786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1894.855878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1894.894980] LNetError: 83069:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1895.916060] LNet: Removed LNI 192.168.201.55@tcp [ 1895.953307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1896.025346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 7 [ 1896.069532] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1897.106351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1898.654641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1898.685915] LNet: 83224:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1898.692233] LNet: 83224:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 1899.751189] LNet: Removed LNI 192.168.201.55@tcp [ 1899.833085] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1899.907047] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1899.931690] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1901.187509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 8 [ 1901.388970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1901.828869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1901.874546] LNetError: 83371:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1901.888673] LNetError: 83371:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1901.912395] LNet: Removed LNI 192.168.201.55@tcp [ 1901.973519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1902.119631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 9 [ 1902.183471] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1903.424489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1904.914731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1906.044340] LNet: Removed LNI 192.168.201.55@tcp [ 1906.074817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1906.142989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1906.183606] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1907.146601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 10 [ 1907.271914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1907.624707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1907.667428] LNet: 83673:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1907.685501] LNet: 83673:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 1908.780367] LNet: Removed LNI 192.168.201.55@tcp [ 1908.825335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1908.903540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 11 [ 1908.933481] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1910.671577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1911.014167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1911.037658] LNetError: 83819:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1911.051182] LNetError: 83819:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 1911.067487] LNet: Removed LNI 192.168.201.55@tcp [ 1911.094766] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1911.236656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1911.277233] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1912.577701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 12 [ 1912.646313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1914.095305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1915.182472] LNet: Removed LNI 192.168.201.55@tcp [ 1915.302918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1915.351886] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 13 [ 1915.389601] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1916.070269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1916.481194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1917.558725] LNet: Removed LNI 192.168.201.55@tcp [ 1917.632860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1917.791591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1917.851425] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1918.619943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 14 [ 1918.714989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1920.126596] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1921.195673] LNet: Removed LNI 192.168.201.55@tcp [ 1921.221530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1921.370352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 15 [ 1921.393531] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1922.438773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 1922.752632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1923.822917] LNet: Removed LNI 192.168.201.55@tcp [ 1923.886263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1923.952162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1923.987705] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1925.182039] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 128 [ 1925.286904] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1925.323515] LNet: 84556:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1925.333517] LNet: 84556:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 1925.360136] LNet: Removed LNI 192.168.201.55@tcp [ 1925.408742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1925.522965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1925.572300] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1926.884329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer -1 [ 1927.060941] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1927.095790] LNetError: 84693:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1927.107678] LNetError: 84693:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 1927.138299] LNet: Removed LNI 192.168.201.55@tcp [ 1927.164955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1927.255399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 128 [ 1927.367319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1927.458711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1927.578242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 1000000000000000000000000 [ 1927.704714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1927.768746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1927.884768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer foo [ 1927.992695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1928.048171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1928.116281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 8 [ 1928.147249] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1929.349106] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1929.436528] LNet: Removed LNI 192.168.201.55@tcp [ 1929.482683] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1929.601922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 1939.029669] Lustre: DEBUG MARKER: == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 09:39:03 (1789133943) [ 1939.593316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1939.649794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1939.770035] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 [ 1939.817623] LNet: Added LNI 192.168.201.55@tcp231 [8/256/0/180] [ 1941.852595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1941.908857] LNet: Removed LNI 192.168.201.55@tcp231 [ 1941.961871] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1942.102789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1942.162390] LNet: Added LNI 192.168.201.55@tcp231 [8/256/0/180] [ 1942.177958] LNet: 85592:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1942.855733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1942.897977] LNet: Removed LNI 192.168.201.55@tcp231 [ 1942.920707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=-1 [ 1942.967665] LNet: Added LNI 192.168.201.55@tcp231 [8/256/0/180] [ 1944.451531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1944.482550] LNet: Removed LNI 192.168.201.55@tcp231 [ 1944.537727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1944.681598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1944.734057] LNet: Added LNI 192.168.201.55@tcp231 [8/256/0/0] [ 1944.743647] LNet: 85745:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1945.307693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1945.371663] LNet: Removed LNI 192.168.201.55@tcp231 [ 1945.389772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=0 [ 1945.421124] LNet: Added LNI 192.168.201.55@tcp231 [8/256/0/0] [ 1947.394016] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1947.467808] LNet: Removed LNI 192.168.201.55@tcp231 [ 1947.497395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1947.616319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1947.643802] LNet: Added LNI 192.168.201.55@tcp231 [8/256/0/60] [ 1947.657628] LNet: 85896:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1948.678744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1948.762347] LNet: Removed LNI 192.168.201.55@tcp231 [ 1948.810779] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=60 [ 1948.848625] LNet: Added LNI 192.168.201.55@tcp231 [8/256/0/60] [ 1949.978629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1950.028733] LNet: Removed LNI 192.168.201.55@tcp231 [ 1950.085302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1950.208723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 1950.257469] LNet: Added LNI 192.168.201.55@tcp231 [8/256/0/180] [ 1950.261957] LNet: 86048:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1951.170214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 1951.230084] LNet: Removed LNI 192.168.201.55@tcp231 [ 1951.276420] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=180 [ 1951.346576] LNet: Added LNI 192.168.201.55@tcp231 [8/256/0/180] [ 1961.806788] Lustre: DEBUG MARKER: == sanity-lnet test 232: Test setting ToS value ========== 09:39:26 (1789133966) [ 1962.124904] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1962.149175] LNet: 86760:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1962.157165] LNet: 86760:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 10 previous similar messages [ 1962.166942] LNetError: 86760:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1962.171230] LNetError: 86760:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 1962.197156] LNet: Removed LNI 192.168.201.55@tcp231 [ 1962.255684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1962.357278] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 [ 1962.370036] LNet: Added LNI 192.168.201.55@tcp232 [8/256/0/180] [ 1963.058453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1963.105328] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1963.136215] LNet: Removed LNI 192.168.201.55@tcp232 [ 1963.163259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos -1 [ 1963.209386] LNet: Added LNI 192.168.201.55@tcp232 [8/256/0/180] [ 1963.935530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1964.003841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1964.052594] LNet: Removed LNI 192.168.201.55@tcp232 [ 1964.079578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 104 [ 1964.114502] LNet: Added LNI 192.168.201.55@tcp232 [8/256/0/180] [ 1964.859599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1964.933741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1964.996278] LNet: Removed LNI 192.168.201.55@tcp232 [ 1965.033472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 106 [ 1965.091404] LNet: Added LNI 192.168.201.55@tcp232 [8/256/0/180] [ 1966.017146] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 1966.048392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 1966.078148] LNet: Removed LNI 192.168.201.55@tcp232 [ 1966.097085] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 300 [ 1973.257143] Lustre: DEBUG MARKER: == sanity-lnet test 233: Check for successful resends ==== 09:39:38 (1789133978) [ 1974.618971] Key type .llcrypt unregistered [ 1974.621247] Key type ._llcrypt unregistered [ 1984.055308] Key type ._llcrypt registered [ 1984.057672] Key type .llcrypt registered [ 1984.338627] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1984.364284] alg: No test for adler32 (adler32-zlib) [ 1985.379335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1985.458592] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 1993.806819] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 2008.271356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2008.366724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2008.414101] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 2010.935584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_interrupt -r 2 [ 2010.995467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e local_interrupt -r 2 [ 2011.115790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_interrupt -r 2 [ 2011.215190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e local_interrupt -r 2 [ 2011.291741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e local_interrupt -r 2 [ 2011.368549] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e local_interrupt -r 2 [ 2011.524245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e local_interrupt -r 2 [ 2011.581906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e local_interrupt -r 2 [ 2011.686746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.55@tcp 192.168.201.155@tcp [ 2011.758287] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 2011.764870] LNet: 1 local NIs in recovery (showing 1): 192.168.201.55@tcp [ 2012.408883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2012.467812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2012.541072] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2012.647303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2012.805908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2012.924898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2013.026798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2013.147067] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2013.199494] LNet: 88558:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2013.256678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2013.300576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2013.332137] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2013.371097] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2013.406374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2013.448113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2014.261440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_dropped -r 2 [ 2014.314610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e local_dropped -r 2 [ 2014.365126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_dropped -r 2 [ 2014.443860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e local_dropped -r 2 [ 2014.504584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e local_dropped -r 2 [ 2014.548119] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e local_dropped -r 2 [ 2014.635507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e local_dropped -r 2 [ 2014.759202] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e local_dropped -r 2 [ 2014.799169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.55@tcp 192.168.201.155@tcp [ 2014.813583] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 2015.697180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2015.758623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2015.913716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2016.082627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2016.182062] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2016.236265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2016.265207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2016.287764] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2016.300376] LNet: 88679:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2016.313266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2016.335140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2016.363263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2016.383502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2016.405693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2016.429677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2016.996739] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_aborted -r 2 [ 2017.041373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e local_aborted -r 2 [ 2017.077570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_aborted -r 2 [ 2017.135573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e local_aborted -r 2 [ 2017.174439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e local_aborted -r 2 [ 2017.205492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e local_aborted -r 2 [ 2017.231779] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e local_aborted -r 2 [ 2017.262540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e local_aborted -r 2 [ 2017.291787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.55@tcp 192.168.201.155@tcp [ 2017.311439] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 2017.549575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2017.579316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2017.607743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2017.636594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2017.671167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2017.705305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2017.738657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2017.776429] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2017.791833] LNet: 88799:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2017.814694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2017.855270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2017.891428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2017.929635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2017.976137] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2018.019854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2018.711910] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_no_route -r 2 [ 2018.784431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e local_no_route -r 2 [ 2018.881551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_no_route -r 2 [ 2018.924666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e local_no_route -r 2 [ 2018.991653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e local_no_route -r 2 [ 2019.102314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e local_no_route -r 2 [ 2019.182551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e local_no_route -r 2 [ 2019.227366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e local_no_route -r 2 [ 2019.312533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.55@tcp 192.168.201.155@tcp [ 2019.344327] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 2019.713542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2019.777127] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2019.832887] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2019.889750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2019.943728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2020.010254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2020.087626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2020.130350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2020.144603] LNet: 88919:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2020.160225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2020.197573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2020.254452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2020.295080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2020.361821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2020.423634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2021.245969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e local_timeout -r 2 [ 2021.304672] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e local_timeout -r 2 [ 2021.356600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e local_timeout -r 2 [ 2021.415425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e local_timeout -r 2 [ 2021.446694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e local_timeout -r 2 [ 2021.479645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e local_timeout -r 2 [ 2021.525279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e local_timeout -r 2 [ 2021.614479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e local_timeout -r 2 [ 2021.668242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.55@tcp 192.168.201.155@tcp [ 2022.009304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2022.049053] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2022.093312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2022.157320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2022.186732] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2022.211622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2022.251784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2022.329707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2022.377570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2022.445118] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2022.515358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2022.575406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2022.668282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2022.702451] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2023.235314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp -m GET -e remote_dropped -r 2 [ 2023.275342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.155@tcp1 -m GET -e remote_dropped -r 2 [ 2023.302374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp -m GET -e remote_dropped -r 2 [ 2023.329172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.201.55@tcp -d 192.168.201.55@tcp1 -m GET -e remote_dropped -r 2 [ 2023.354852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp -m GET -e remote_dropped -r 2 [ 2023.382733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.155@tcp1 -m GET -e remote_dropped -r 2 [ 2023.416986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp -m GET -e remote_dropped -r 2 [ 2023.440667] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.201.55@tcp1 -d 192.168.201.55@tcp1 -m GET -e remote_dropped -r 2 [ 2023.458476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.201.55@tcp 192.168.201.155@tcp [ 2023.472501] LNet: There was an unexpected network error while writing to 192.168.201.155: rc = -22 [ 2023.475100] LNet: Skipped 1 previous similar message [ 2023.476984] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.155@tcp [ 2023.603795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2023.639934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2023.675338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2023.747803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2023.785517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2023.881062] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2023.934622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2023.957828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2024.050884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2024.125496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2024.201619] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2024.255140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2024.290411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2024.330409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2033.013489] LNet: 89239:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2033.035432] LNetError: 89239:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2034.083082] LNet: Removed LNI 192.168.201.55@tcp [ 2034.098189] LNetError: 89239:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2034.135360] LNet: Removed LNI 192.168.201.55@tcp1 [ 2034.997798] Key type .llcrypt unregistered [ 2034.999485] Key type ._llcrypt unregistered [ 2058.503389] Key type ._llcrypt registered [ 2058.506758] Key type .llcrypt registered [ 2060.624546] Lustre: DEBUG MARKER: == sanity-lnet test 234: Verify NI status changes with manual health value transitions ========================================================== 09:41:04 (1789134064) [ 2061.540262] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2061.571461] alg: No test for adler32 (adler32-zlib) [ 2062.738419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2062.775830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2062.795958] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2063.998616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 0 [ 2064.041236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.201.55@tcp [ 2064.075447] LNetError: 90220:0:(api-ni.c:4473:lnet_ni_set_healthv()) manually adding local NI 192.168.201.55@tcp to recovery [ 2064.203482] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --nid 192.168.201.55@tcp [ 2064.368114] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2074.090856] Lustre: DEBUG MARKER: == sanity-lnet test 235: Verify NI status DOWN when health reaches 0 after failures ========================================================== 09:41:18 (1789134078) [ 2074.254346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2074.285647] LNet: 90805:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2074.307413] LNetError: 90805:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2074.336461] LNet: Removed LNI 192.168.201.55@tcp [ 2074.392563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2074.521454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2074.570448] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2076.115535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2076.143615] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 2076.935601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 100 --nid 192.168.201.55@tcp [ 2076.951726] LNetError: 91078:0:(api-ni.c:4473:lnet_ni_set_healthv()) manually adding local NI 192.168.201.55@tcp to recovery [ 2077.536600] LNet: 1 local NIs in recovery (showing 1): 192.168.201.55@tcp [ 2077.570349] LNet: There was an unexpected network error while writing to 192.168.201.55: rc = -22 [ 2078.243476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 900 --nid 192.168.201.55@tcp [ 2087.793418] Lustre: DEBUG MARKER: == sanity-lnet test 236: Local NI state propagates to routers ========================================================== 09:41:32 (1789134092) [ 2089.909752] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_236 Need at least 2 remote nodes found oleg155-server [ 2091.536529] Lustre: DEBUG MARKER: == sanity-lnet test 237: Check MR forwarding feature ===== 09:41:36 (1789134096) [ 2093.660776] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_237 Need at least 2 remote nodes found oleg155-server [ 2095.973205] Lustre: DEBUG MARKER: == sanity-lnet test 238: Check MR forwarding feature skipped for non-MR source ========================================================== 09:41:40 (1789134100) [ 2098.580966] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_238 Need at least 2 remote nodes found oleg155-server [ 2100.461481] Lustre: DEBUG MARKER: == sanity-lnet test 241: Check setting LND timeout value via lnetctl updates tunables ========================================================== 09:41:45 (1789134105) [ 2100.728855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2100.761236] LNet: 92709:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2100.767468] LNetError: 92709:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2101.795198] LNet: Removed LNI 192.168.201.55@tcp [ 2101.799272] LNetError: 92709:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2101.823073] LNet: Removed LNI 192.168.201.55@tcp1 [ 2101.886945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2102.009492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2102.043220] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2102.898350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set -h [ 2103.031893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 51 [ 2103.105296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 2103.228537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 0 [ 2103.292513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 2103.336759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 50 [ 2110.037368] Lustre: DEBUG MARKER: == sanity-lnet test 245: Check read-only params do not appear in export --backup ========================================================== 09:41:55 (1789134115) [ 2110.117923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2110.168736] LNet: 93427:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2110.187081] LNetError: 93427:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2110.215985] LNet: Removed LNI 192.168.201.55@tcp [ 2110.247651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2110.354136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2110.374061] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2117.937263] Lustre: DEBUG MARKER: == sanity-lnet test 250: test that linux routes are added ========================================================== 09:42:02 (1789134122) [ 2118.194912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2118.211953] LNet: 94138:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2118.216977] LNetError: 94138:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2118.229539] LNet: Removed LNI 192.168.201.55@tcp [ 2118.247299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2118.305659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2118.349534] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2127.506508] Lustre: DEBUG MARKER: == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 09:42:11 (1789134131) [ 2129.012489] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_251 Need kfi NETTYPE [ 2130.935806] Lustre: DEBUG MARKER: == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 09:42:15 (1789134135) [ 2131.358617] LNet: 95174:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2131.363476] LNetError: 95174:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2131.375680] LNet: Removed LNI 192.168.201.55@tcp [ 2132.182634] Key type .llcrypt unregistered [ 2132.186418] Key type ._llcrypt unregistered [ 2140.607700] Key type ._llcrypt registered [ 2140.609441] Key type .llcrypt registered [ 2140.860732] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2140.893709] alg: No test for adler32 (adler32-zlib) [ 2141.844506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2141.874946] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2150.061328] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 2154.717983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --timeout 15 192.168.201.155@tcp [ 2155.489413] LNet: 95601:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2155.495301] LNetError: 95601:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2155.514845] LNet: Removed LNI 192.168.201.55@tcp [ 2155.871737] Key type .llcrypt unregistered [ 2155.875492] Key type ._llcrypt unregistered [ 2173.156467] Key type ._llcrypt registered [ 2173.162603] Key type .llcrypt registered [ 2175.696259] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253 [ 2178.224734] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254 [ 2179.802344] Lustre: DEBUG MARKER: == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 09:43:05 (1789134185) [ 2182.616973] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_255 kernel ip2nets deprecated in 2.17.53 [ 2185.606381] Lustre: DEBUG MARKER: == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 09:43:09 (1789134189) [ 2188.656618] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_256 Need at least 2 remote nodes found oleg155-server [ 2189.967430] Lustre: DEBUG MARKER: == sanity-lnet test 257: Test graceful router shutdown/startup ========================================================== 09:43:15 (1789134195) [ 2192.355087] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_257 Need at least 3 remote nodes found oleg155-server [ 2194.256387] Lustre: DEBUG MARKER: == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 09:43:19 (1789134199) [ 2196.112465] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2196.438474] alg: No test for adler32 (adler32-zlib) [ 2197.681924] Lustre: Lustre: Build Version: 2.17.57_131_gc13dec9 [ 2197.959868] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2199.616168] Key type lgssc registered [ 2200.787152] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2222.871384] Key type lgssc unregistered [ 2223.234437] LNet: 98785:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2223.242835] LNetError: 98785:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2223.264237] LNet: Removed LNI 192.168.201.55@tcp [ 2224.149647] Key type .llcrypt unregistered [ 2224.151488] Key type ._llcrypt unregistered [ 2241.896835] Key type ._llcrypt registered [ 2241.901694] Key type .llcrypt registered [ 2243.733698] Lustre: DEBUG MARKER: == sanity-lnet test 265: Import of invalid CPT should fail ========================================================== 09:44:08 (1789134248) [ 2244.214729] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2244.233443] alg: No test for adler32 (adler32-zlib) [ 2245.101595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2245.168288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2245.203289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2245.236932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2245.284854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 2252.629342] Lustre: DEBUG MARKER: == sanity-lnet test 266: Validate CPT parsing in network sequence ========================================================== 09:44:17 (1789134257) [ 2252.934324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2252.988090] LNet: 100203:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2253.017497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2253.111923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2253.145075] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2254.000566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2254.019394] LNet: 100347:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2254.024746] LNetError: 100347:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2254.040772] LNet: Removed LNI 192.168.201.55@tcp [ 2254.055987] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2254.104460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 2254.126443] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2254.491970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2254.531891] LNet: Removed LNI 192.168.201.55@tcp [ 2254.546143] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2254.598744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 2254.638331] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2261.470319] Lustre: DEBUG MARKER: == sanity-lnet test 270: Test NID metadata =============== 09:44:26 (1789134266) [ 2261.856971] LNet: 100966:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2261.865433] LNet: 100966:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 2261.876996] LNetError: 100966:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2261.894459] LNetError: 100966:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2261.915879] LNet: Removed LNI 192.168.201.55@tcp [ 2262.795687] Key type .llcrypt unregistered [ 2262.801626] Key type ._llcrypt unregistered [ 2269.919728] Key type ._llcrypt registered [ 2269.932396] Key type .llcrypt registered [ 2270.223910] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2270.242568] alg: No test for adler32 (adler32-zlib) [ 2271.299734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2271.401083] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2271.408189] LNet: *** cfs_fail_loc=e003, val=100*** [ 2272.295449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2272.317665] LNet: *** cfs_fail_loc=e003, val=100*** [ 2272.319994] LNet: Skipped 2 previous similar messages [ 2272.324937] Ping reply received with mapping: [ 2272.326484] found metadata reply NI[192.168.201.55@tcp] [ 2272.329781] 37 c9 a8 c0 00 00 02 00 5a 5a 5a 5a 5a 5a 5a 5a [ 2272.333653] 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a [ 2272.337485] 5a 5a 5a 5a 5a 5a 5a 5a [ 2272.486608] LNet: 101368:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2272.491219] LNetError: 101368:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2273.529226] LNet: Removed LNI 192.168.201.55@tcp [ 2274.122776] Key type .llcrypt unregistered [ 2274.125643] Key type ._llcrypt unregistered [ 2286.245356] Key type ._llcrypt registered [ 2286.248318] Key type .llcrypt registered [ 2287.984596] Lustre: DEBUG MARKER: == sanity-lnet test 280: Don't panic when request_module fails ========================================================== 09:44:52 (1789134292) [ 2288.630552] Key type .llcrypt unregistered [ 2288.633315] Key type ._llcrypt unregistered [ 2295.233459] Key type ._llcrypt registered [ 2295.235218] Key type .llcrypt registered [ 2295.609652] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2295.657525] alg: No test for adler32 (adler32-zlib) [ 2296.816304] LNetError: 102426:0:(api-ni.c:2850:lnet_load_lnd()) Can't load LND tcp, module ksocklnd, rc=256 [ 2297.229878] Key type .llcrypt unregistered [ 2297.231610] Key type ._llcrypt unregistered [ 2303.762688] Key type ._llcrypt registered [ 2303.765562] Key type .llcrypt registered [ 2305.219685] Lustre: DEBUG MARKER: == sanity-lnet test 290: Check that lnet acceptor is using explicit address binding ========================================================== 09:45:09 (1789134309) [ 2305.792520] Key type .llcrypt unregistered [ 2305.794119] Key type ._llcrypt unregistered [ 2313.870324] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2313.873718] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2315.637393] Key type ._llcrypt registered [ 2315.639693] Key type .llcrypt registered [ 2315.830453] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2315.860587] alg: No test for adler32 (adler32-zlib) [ 2316.840609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2316.895587] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2327.806562] Lustre: DEBUG MARKER: == sanity-lnet test 291: load lnet using custom accept port for bulk tcp connections ========================================================== 09:45:32 (1789134332) [ 2328.202542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2328.251832] LNet: 104208:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2328.259858] LNetError: 104208:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2328.284118] LNet: Removed LNI 10.1.2.3@tcp [ 2328.337197] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2329.192323] Key type .llcrypt unregistered [ 2329.195226] Key type ._llcrypt unregistered [ 2338.592037] Key type ._llcrypt registered [ 2338.594609] Key type .llcrypt registered [ 2338.824354] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2338.872791] alg: No test for adler32 (adler32-zlib) [ 2339.880709] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2339.944673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2340.066578] LNet: 104502:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2340.073985] LNetError: 104502:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2340.095670] LNet: Removed LNI 192.168.201.55@tcp [ 2340.957431] Key type .llcrypt unregistered [ 2340.959169] Key type ._llcrypt unregistered [ 2358.352980] Key type ._llcrypt registered [ 2358.357584] Key type .llcrypt registered [ 2360.264822] Lustre: DEBUG MARKER: == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 09:46:04 (1789134364) [ 2361.587465] Key type .llcrypt unregistered [ 2361.589226] Key type ._llcrypt unregistered [ 2369.933697] Key type ._llcrypt registered [ 2369.940818] Key type .llcrypt registered [ 2370.416065] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2370.443357] alg: No test for adler32 (adler32-zlib) [ 2380.293724] Lustre: DEBUG MARKER: == sanity-lnet test 301: Check for dynamic adds of same/wrong interface (memory leak) ========================================================== 09:46:24 (1789134384) [ 2380.410839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2380.472929] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2380.573269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2380.651366] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2381.622200] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2381.649724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if ens2 [ 2381.683051] LNet: Removed LNI 192.168.201.55@tcp [ 2381.957367] LNet: 106260:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2382.339552] Key type .llcrypt unregistered [ 2382.341249] Key type ._llcrypt unregistered [ 2394.083950] Key type ._llcrypt registered [ 2394.090706] Key type .llcrypt registered [ 2396.191574] Lustre: DEBUG MARKER: == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 09:46:40 (1789134400) [ 2396.772761] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2396.795779] alg: No test for adler32 (adler32-zlib) [ 2397.912369] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2398.018386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2398.081243] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2398.969424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2399.015430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug peer --nid 192.168.201.55@tcp [ 2399.040499] LNet: Conn 0 [type, closing, crefcnt, srefcnt]: 2, 0, 1, 1 [ 2399.042576] LNet: Conn 0 rx [scheduled, ready, state]: 0, 0, 1 [ 2399.047814] LNet: Conn 0 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2398, 0, 0 [ 2399.051592] LNet: Conn 0 sched [nconns, cpt]: 4, 0 [ 2399.053512] LNet: Conn 1 [type, closing, crefcnt, srefcnt]: 3, 0, 1, 1 [ 2399.055473] LNet: Conn 1 rx [scheduled, ready, state]: 0, 0, 1 [ 2399.058457] LNet: Conn 1 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2398, 0, 0 [ 2399.064228] LNet: Conn 1 sched [nconns, cpt]: 4, 0 [ 2399.067569] LNet: Conn 2 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 2399.071629] LNet: Conn 2 rx [scheduled, ready, state]: 0, 0, 1 [ 2399.078180] LNet: Conn 2 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2398, 0, 2448 [ 2399.087314] LNet: Conn 2 sched [nconns, cpt]: 4, 0 [ 2399.095708] LNet: Conn 3 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 2399.102967] LNet: Conn 3 rx [scheduled, ready, state]: 0, 0, 1 [ 2399.110886] LNet: Conn 3 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 2398, 0, 2448 [ 2399.118097] LNet: Conn 3 sched [nconns, cpt]: 4, 0 [ 2408.295340] Lustre: DEBUG MARKER: == sanity-lnet test 303: Check peer NI health after link down ========================================================== 09:46:52 (1789134412) [ 2408.938392] LNet: 107801:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2408.947469] LNetError: 107801:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2408.973557] LNet: Removed LNI 192.168.201.55@tcp [ 2410.225690] Key type .llcrypt unregistered [ 2410.229480] Key type ._llcrypt unregistered [ 2420.782305] Key type ._llcrypt registered [ 2420.784424] Key type .llcrypt registered [ 2421.202873] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2421.248150] alg: No test for adler32 (adler32-zlib) [ 2422.291255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2422.354352] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2429.779351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 2439.331498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2439.377184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2439.404865] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 2441.004058] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 2441.189719] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2441.194579] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2441.197590] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 2443.847463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp99 --if test1pg [ 2443.875403] LNet: Added LNI 10.1.2.3@tcp99 [8/256/0/180] [ 2455.842410] Lustre: DEBUG MARKER: == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 09:47:40 (1789134460) [ 2456.209223] LNet: 109313:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2456.223242] LNetError: 109313:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2456.231772] LNet: Removed LNI 192.168.201.55@tcp [ 2457.248192] LNet: Removed LNI 192.168.201.55@tcp1 [ 2457.252622] LNetError: 109313:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2457.264758] LNetError: 109313:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2457.285672] LNet: Removed LNI 10.1.2.3@tcp99 [ 2457.956719] Key type .llcrypt unregistered [ 2457.958438] Key type ._llcrypt unregistered [ 2466.105823] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2466.119626] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2468.823384] Key type ._llcrypt registered [ 2468.832525] Key type .llcrypt registered [ 2469.114311] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2469.135394] alg: No test for adler32 (adler32-zlib) [ 2470.124566] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2470.179817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2470.209401] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2470.884707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2470.917887] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2473.359479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.1.2.3@tcp --lock_prim [ 2473.423217] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2473.472171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 10.1.2.3@tcp --state 1048577 [ 2473.505122] LNetError: 110062:0:(peer.c:2362:lnet_del_peer_ni()) peer 10.1.2.3@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 2473.550744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2473.626316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2474.208425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 2474.305350] LNet: 110082:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2474.316794] LNetError: 110082:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2474.324218] LNet: Removed LNI 10.1.2.3@tcp [ 2475.371831] LNet: Removed LNI 192.168.201.55@tcp [ 2476.029446] Key type .llcrypt unregistered [ 2476.035358] Key type ._llcrypt unregistered [ 2490.968930] Key type ._llcrypt registered [ 2490.971981] Key type .llcrypt registered [ 2492.072709] Lustre: DEBUG MARKER: == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 09:48:17 (1789134497) [ 2492.557501] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2492.571684] alg: No test for adler32 (adler32-zlib) [ 2493.537307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2493.636598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2493.683114] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2494.572989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2494.676454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping oleg155-client@tcp [ 2502.895356] Lustre: DEBUG MARKER: == sanity-lnet test 310: Set timeout and verify ========== 09:48:27 (1789134507) [ 2503.358472] LNet: 111622:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2503.367995] LNetError: 111622:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2503.389693] LNet: Removed LNI 192.168.201.55@tcp [ 2504.424648] Key type .llcrypt unregistered [ 2504.429722] Key type ._llcrypt unregistered [ 2514.358972] Key type ._llcrypt registered [ 2514.361731] Key type .llcrypt registered [ 2514.513510] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2514.534903] alg: No test for adler32 (adler32-zlib) [ 2515.468428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure --all [ 2515.484611] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2515.510804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 2525.079719] Lustre: DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 09:48:48 (1789134528) [ 2526.848606] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type [ 2529.137933] Lustre: DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 09:48:53 (1789134533) [ 2531.467089] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type [ 2533.286216] Lustre: DEBUG MARKER: == sanity-lnet test 350: Check import --exec of ping/discover ========================================================== 09:48:58 (1789134538) [ 2533.545137] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2533.592355] LNet: 113046:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2533.603236] LNetError: 113046:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2533.624914] LNet: Removed LNI 192.168.201.55@tcp [ 2533.684250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2533.804782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2533.863438] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2535.013330] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 2535.072606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 2543.262448] Lustre: DEBUG MARKER: == sanity-lnet test 400: Check for udsp add/delete net rule without net num ========================================================== 09:49:07 (1789134547) [ 2543.555659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2543.591564] LNet: 113756:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2543.603290] LNetError: 113756:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2543.615323] LNet: Removed LNI 192.168.201.55@tcp [ 2543.654275] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2543.780823] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --priority 0 [ 2543.839714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2545.372492] Key type .llcrypt unregistered [ 2545.379794] Key type ._llcrypt unregistered [ 2560.498580] Key type ._llcrypt registered [ 2560.502347] Key type .llcrypt registered [ 2562.040272] Lustre: DEBUG MARKER: == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 09:49:26 (1789134566) [ 2562.452947] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2562.486605] alg: No test for adler32 (adler32-zlib) [ 2563.396909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2563.457156] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2563.496864] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2564.259426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2564.358958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2572.641333] Lustre: DEBUG MARKER: == sanity-lnet test 402: Destination net rule should not panic ========================================================== 09:49:37 (1789134577) [ 2572.740639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2572.778175] LNet: 115313:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2572.791365] LNetError: 115313:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2572.817732] LNet: Removed LNI 192.168.201.55@tcp [ 2572.831810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2572.878224] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst kfi --priority 0 [ 2572.915912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 402@kfi [ 2582.029646] Lustre: DEBUG MARKER: == sanity-lnet test 403: Add and delete udsp net and nid rules ========================================================== 09:49:46 (1789134586) [ 2582.360256] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2582.407851] LNet: 115899:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2582.431326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2582.527850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2583.040503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2583.089544] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2583.202457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src 1.1.1.1@tcp --prio 1 [ 2584.120100] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2584.191026] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2584.327587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2584.362561] LNet: 115933:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2584.400991] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2584.573447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2585.269551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2585.345921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2585.489433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst 1.1.1.1@tcp --prio 1 [ 2586.163161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 2586.227712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 2593.074847] Lustre: DEBUG MARKER: == sanity-lnet test 404: Check udsp src/dst net rule ===== 09:49:58 (1789134598) [ 2593.470968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2593.496650] LNet: 116533:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2593.554491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2593.635538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2593.695703] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2594.517981] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2594.548114] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2595.732969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2595.803970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2596.180405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2596.223443] LNetError: 116813:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2596.241236] LNet: Removed LNI 192.168.201.55@tcp [ 2596.260980] LNet: Removed LNI 192.168.201.55@tcp2 [ 2596.319292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2596.413531] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2596.426662] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2596.436588] LNet: 116819:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2596.924949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2597.215633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2597.259525] LNetError: 116856:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2597.270953] LNetError: 116856:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2597.285571] LNet: Removed LNI 192.168.201.55@tcp [ 2597.292886] LNet: Removed LNI 192.168.201.55@tcp2 [ 2597.322146] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2597.406456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2597.452965] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2598.305963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2598.325368] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2599.248399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2599.274734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 2599.303536] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2599.452428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2599.465916] LNet: 117141:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2599.470046] LNet: 117141:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 2599.476053] LNetError: 117141:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2599.480254] LNetError: 117141:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 2599.485906] LNet: Removed LNI 192.168.201.55@tcp [ 2600.562266] LNet: Removed LNI 192.168.201.55@tcp2 [ 2600.610428] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2600.734896] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2600.744741] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2600.749245] LNet: 117147:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2601.122501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2601.280744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2601.304833] LNet: Removed LNI 192.168.201.55@tcp [ 2602.344406] LNet: Removed LNI 192.168.201.55@tcp2 [ 2602.421907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2602.527656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2602.566037] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2603.539413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2603.562697] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2604.514592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2604.576243] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2604.900787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2604.926107] LNetError: 117464:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2604.931969] LNetError: 117464:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 2605.984195] LNet: Removed LNI 192.168.201.55@tcp [ 2605.995322] LNet: Removed LNI 192.168.201.55@tcp2 [ 2606.016293] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2606.061231] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2606.065600] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2606.070723] LNet: 117470:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2606.662984] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2606.897792] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2607.970862] LNet: Removed LNI 192.168.201.55@tcp [ 2608.002279] LNet: Removed LNI 192.168.201.55@tcp2 [ 2608.068742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2608.206272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2608.260416] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2609.037403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2609.064651] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2610.251892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2610.280727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 2610.334196] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2610.547656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2610.575849] LNet: 117792:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2610.580672] LNet: 117792:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 2611.616376] LNet: Removed LNI 192.168.201.55@tcp [ 2612.646595] LNet: Removed LNI 192.168.201.55@tcp2 [ 2612.703642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2612.805509] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2612.821593] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2612.830232] LNet: 117798:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2613.591245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2613.931241] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2613.964465] LNetError: 117835:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2613.976733] LNetError: 117835:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 2613.995510] LNet: Removed LNI 192.168.201.55@tcp [ 2615.031855] LNet: Removed LNI 192.168.201.55@tcp2 [ 2615.073695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2615.200198] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2615.234325] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2616.262996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2616.293755] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2617.560806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 2617.622575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 2617.738626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2618.251283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2619.296165] LNet: Removed LNI 192.168.201.55@tcp [ 2619.309493] LNet: Removed LNI 192.168.201.55@tcp2 [ 2619.362486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2619.459786] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2619.464781] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2619.470908] LNet: 118138:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2620.152942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2620.650957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2621.728186] LNet: Removed LNI 192.168.201.55@tcp [ 2621.739070] LNet: Removed LNI 192.168.201.55@tcp2 [ 2621.833661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2621.930347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2621.987183] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2623.049919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2623.087627] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2624.171812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2624.221589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 2624.252320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 2624.304800] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2624.777462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2625.824815] LNet: Removed LNI 192.168.201.55@tcp [ 2626.856200] LNet: Removed LNI 192.168.201.55@tcp2 [ 2626.894389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2626.941255] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2626.945599] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2627.477591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2636.510949] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_410 skipping ALWAYS excluded test 410 [ 2638.948163] Lustre: DEBUG MARKER: == sanity-lnet test 450: Check import of multiple interfaces ========================================================== 09:50:43 (1789134643) [ 2639.249239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2639.294463] LNet: 119258:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2639.299466] LNet: 119258:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 4 previous similar messages [ 2639.305214] LNetError: 119258:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2639.311454] LNetError: 119258:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 7 previous similar messages [ 2639.317817] LNet: Removed LNI 192.168.201.55@tcp [ 2640.362150] LNet: Removed LNI 192.168.201.55@tcp2 [ 2640.424368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2640.981765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2641.057551] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2642.366258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2642.402518] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2643.439354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2643.495807] LNet: Removed LNI 10.1.2.3@tcp [ 2643.523466] LNet: Removed LNI 192.168.201.55@tcp [ 2643.575608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2643.686774] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-450.yaml [ 2643.775066] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2643.800829] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2644.598986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del /tmp/sanity-lnet-450.yaml [ 2644.620652] LNet: Removed LNI 192.168.201.55@tcp [ 2644.639232] LNet: Removed LNI 10.1.2.3@tcp [ 2653.632907] Lustre: DEBUG MARKER: == sanity-lnet test 475: per-operation latency stats populate and reset ========================================================== 09:50:58 (1789134658) [ 2653.814521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2653.899295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2653.988432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2654.012269] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2654.865678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 2654.919225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2654.961630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2655.007846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2655.050755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2655.097264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2655.155870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2655.220532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2655.249266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2655.302810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2655.343246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.201.55@tcp [ 2655.545875] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl stats reset [ 2662.338757] Lustre: DEBUG MARKER: == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 09:51:07 (1789134667) [ 2662.592593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2663.655826] LNet: Removed LNI 192.168.201.55@tcp [ 2663.729414] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2663.974692] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 2664.127107] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2664.130879] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2664.145582] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 2666.558197] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2666.604513] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2667.425960] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 2667.446830] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 2669.940814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2670.005031] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d *@tcp -r 1 -l 1 -m PUT [ 2670.114332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.201.55@tcp -m PUT -e local_timeout -r 1 [ 2670.200869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 10.1.2.3@tcp -m PUT -e local_timeout -r 1 [ 2670.278880] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 2670.287882] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 2670.680316] LNetError: 120930:0:(lib-move.c:4801:lnet_parse()) 192.168.201.55@tcp, src 192.168.201.55@tcp: Dropping PUT (error -108 looking up sender) [ 2671.713397] LNet: 121394:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2671.729853] LNet: 121394:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 2671.735900] LNetError: 121394:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface test1pg not found [ 2671.757166] LNetError: 121394:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Skipped 4 previous similar messages [ 2671.770510] LNet: Removed LNI 10.1.2.3@tcp [ 2672.807256] LNet: Removed LNI 192.168.201.55@tcp [ 2673.335722] Key type .llcrypt unregistered [ 2673.339686] Key type ._llcrypt unregistered [ 2688.787298] Key type ._llcrypt registered [ 2688.798907] Key type .llcrypt registered [ 2690.117704] Lustre: DEBUG MARKER: == sanity-lnet test 501: Verify lnetctl peer set --health (SR) ========================================================== 09:51:34 (1789134694) [ 2690.895126] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2690.917295] alg: No test for adler32 (adler32-zlib) [ 2691.817798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2692.149386] LNet: 122248:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2692.830467] Key type .llcrypt unregistered [ 2692.833695] Key type ._llcrypt unregistered [ 2700.334209] Key type ._llcrypt registered [ 2700.336345] Key type .llcrypt registered [ 2700.571517] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2700.594482] alg: No test for adler32 (adler32-zlib) [ 2701.552497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2701.593460] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2711.155976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 2711.463687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 2712.019581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.155@tcp [ 2712.250652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2712.282536] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2712.312214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2712.338849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2712.394804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2712.446719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2712.472679] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2712.504858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2712.529788] LNet: 122702:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2712.550512] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2712.592912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2712.628807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2712.650254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2712.673750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2712.701955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2713.387710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 2713.648338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2713.709859] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2713.783179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2713.827111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2713.886529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2713.888864] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.155@tcp [ 2713.950550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2713.999341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2714.056955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2714.077664] LNet: 122819:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2714.117809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2714.186905] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2714.247116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2714.299642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2714.335584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2714.402025] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2715.125893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.155@tcp [ 2715.473555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2715.508690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2715.541902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2715.623565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2715.691592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2715.751266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2715.786210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2715.831227] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2715.855164] LNet: 122936:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2715.876219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2715.934992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2715.973201] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2716.005882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2716.043227] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2716.077143] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2716.697388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 2716.968908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2716.998681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2717.028634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2717.078841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2717.128579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2717.154989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2717.187601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2717.222196] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2717.252789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2717.282444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2717.322294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2717.351284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2717.426190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2717.478148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2717.940564] LNet: 123104:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2717.948327] LNetError: 123104:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2717.973842] LNet: Removed LNI 192.168.201.55@tcp [ 2718.323752] Key type .llcrypt unregistered [ 2718.328133] Key type ._llcrypt unregistered [ 2735.722781] Key type ._llcrypt registered [ 2735.726587] Key type .llcrypt registered [ 2737.433631] Lustre: DEBUG MARKER: == sanity-lnet test 502: Verify lnetctl peer set --health (MR) ========================================================== 09:52:21 (1789134741) [ 2738.090676] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2738.133194] alg: No test for adler32 (adler32-zlib) [ 2739.168406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2739.903458] LNet: 123991:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2740.909469] Key type .llcrypt unregistered [ 2740.915578] Key type ._llcrypt unregistered [ 2749.272292] Key type ._llcrypt registered [ 2749.275563] Key type .llcrypt registered [ 2749.464647] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 2749.478910] alg: No test for adler32 (adler32-zlib) [ 2750.394527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 2750.426642] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2760.148511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.155@tcp [ 2774.116095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2774.222026] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 2774.274964] LNet: Added LNI 192.168.201.55@tcp1 [8/256/0/180] [ 2775.773560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 2776.743243] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.201.155@tcp [ 2777.070253] LNet: 1 peer NIs in recovery (showing 1): 192.168.201.155@tcp [ 2777.326335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2777.404542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2777.501706] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2777.581274] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2777.686803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2777.741853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2777.860967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2777.947618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2778.001928] LNet: 124649:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2778.102828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2778.175657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2778.206977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2778.255961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2778.317476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2778.365965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2779.285466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 2779.646867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2779.699967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2779.766044] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2779.802440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2779.828452] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2779.858466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2779.904206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2779.949943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2779.964730] LNet: 124766:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2779.991377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2780.051322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2780.122491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2780.157559] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2780.192516] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2780.251427] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2781.176945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.201.155@tcp [ 2781.604048] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2781.653827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2781.716524] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2781.774868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2781.804785] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2781.837728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2781.895731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2781.937503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2781.974684] LNet: 124884:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2781.995992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2782.092353] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2782.167836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2782.266993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2782.372237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2782.450546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2783.543878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 2783.978884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 2784.035485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 4096 [ 2784.087433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 2784.149332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 2784.216443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 2784.255456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 2784.314117] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 2784.375305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 2784.414987] LNet: 125001:0:(api-ni.c:296:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 2784.438212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 2784.462306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 2784.530267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 2784.570304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 2784.603668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 2784.632839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 2792.163982] LNet: 125086:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2792.170540] LNetError: 125086:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2793.186437] LNet: Removed LNI 192.168.201.55@tcp [ 2793.195995] LNetError: 125086:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2793.221093] LNet: Removed LNI 192.168.201.55@tcp1 [ 2793.744698] Key type .llcrypt unregistered [ 2793.750348] Key type ._llcrypt unregistered [ 2812.033675] Key type ._llcrypt registered [ 2812.041639] Key type .llcrypt registered [ 2813.905904] Lustre: DEBUG MARKER: == sanity-lnet test 525: keep reachable peer on local net delete ========================================================== 09:53:38 (1789134818) [ 2814.581292] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2814.603928] alg: No test for adler32 (adler32-zlib) [ 2815.670777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2815.723957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2815.761191] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2816.739564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 2816.778628] LNet: Added LNI 192.168.201.55@tcp2 [8/256/0/180] [ 2817.843618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.201.55@tcp [ 2817.963394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp [ 2819.040168] LNet: Removed LNI 192.168.201.55@tcp [ 2826.937340] Lustre: DEBUG MARKER: == sanity-lnet test 550: DD-off: keep route up when gw reachable via another local NI ========================================================== 09:53:51 (1789134831) [ 2828.861585] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_550 Need at least 2 local interfaces for multi-rail [ 2831.053427] Lustre: DEBUG MARKER: == sanity-lnet test 600: lnet_legacy2yaml: basic ip2nets conversion ========================================================== 09:53:55 (1789134835) [ 2842.404929] Lustre: DEBUG MARKER: == sanity-lnet test 601: lnet_legacy2yaml: basic routes conversion ========================================================== 09:54:06 (1789134846) [ 2850.684860] Lustre: DEBUG MARKER: == sanity-lnet test 602: lnet_legacy2yaml: routes with priority ========================================================== 09:54:15 (1789134855) [ 2858.252306] Lustre: DEBUG MARKER: == sanity-lnet test 603: lnet_legacy2yaml: default hop flag ========================================================== 09:54:23 (1789134863) [ 2865.609923] Lustre: DEBUG MARKER: == sanity-lnet test 604: lnet_legacy2yaml: default hop with explicit value ========================================================== 09:54:31 (1789134871) [ 2873.461095] Lustre: DEBUG MARKER: == sanity-lnet test 605: lnet_legacy2yaml: quote leading star in IP pattern ========================================================== 09:54:38 (1789134878) [ 2881.269897] Lustre: DEBUG MARKER: == sanity-lnet test 606: lnet_legacy2yaml: file inputs === 09:54:46 (1789134886) [ 2890.034589] Lustre: DEBUG MARKER: == sanity-lnet test 607: lnet_legacy2yaml: options line with combined ip2nets and routes ========================================================== 09:54:55 (1789134895) [ 2899.005065] Lustre: DEBUG MARKER: == sanity-lnet test 608: lnet_legacy2yaml: error on invalid rule (missing IP pattern) ========================================================== 09:55:03 (1789134903) [ 2906.954405] Lustre: DEBUG MARKER: == sanity-lnet test 609: lnet_legacy2yaml: error on no input ========================================================== 09:55:11 (1789134911) [ 2917.127520] Lustre: DEBUG MARKER: == sanity-lnet test 610: lnet_legacy2yaml: multiple files and inline, check order ========================================================== 09:55:21 (1789134921) [ 2927.586766] Lustre: DEBUG MARKER: == sanity-lnet test 611: lnet_legacy2yaml: invalid hopcount ========================================================== 09:55:32 (1789134932) [ 2936.273096] Lustre: DEBUG MARKER: == sanity-lnet test 612: lnet_legacy2yaml: malformed IP pattern ========================================================== 09:55:40 (1789134940) [ 2949.004924] Lustre: DEBUG MARKER: == sanity-lnet test 613: lnet_legacy2yaml: multiple IP patterns in single rule ========================================================== 09:55:52 (1789134952) [ 2960.646988] Lustre: DEBUG MARKER: == sanity-lnet test 614: lnet_legacy2yaml: interfaces enumeration with multiple IP ranges ========================================================== 09:56:05 (1789134965) [ 2972.228584] Lustre: DEBUG MARKER: == sanity-lnet test 615: lnet_legacy2yaml: modprobe configuration file input ========================================================== 09:56:16 (1789134976) [ 2980.420227] Lustre: DEBUG MARKER: == sanity-lnet test 616: lnet_legacy2yaml: combining modprobe file with explicit ip2nets file ========================================================== 09:56:24 (1789134984) [ 2988.511549] Lustre: DEBUG MARKER: == sanity-lnet test 617: lnet_legacy2yaml: round-trip tunables from LNet to modprobe to YAML ========================================================== 09:56:33 (1789134993) [ 2988.742874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2988.780254] LNet: 136785:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2988.800070] LNetError: 136785:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2989.879695] LNet: Removed LNI 192.168.201.55@tcp2 [ 2989.918492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2990.156026] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 2990.203638] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2991.641432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 2991.684377] LNet: 136928:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2991.696786] LNetError: 136928:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2991.738154] LNet: Removed LNI 192.168.201.55@tcp [ 2991.784393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 2991.851441] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 2991.882465] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 2999.384993] Lustre: DEBUG MARKER: == sanity-lnet test 618: lnet_legacy2yaml: alternative route format ========================================================== 09:56:44 (1789135004) [ 3007.650333] Lustre: DEBUG MARKER: == sanity-lnet test 619: lnet_legacy2yaml: multiple alternative format routes from file ========================================================== 09:56:52 (1789135012) [ 3016.979108] Lustre: DEBUG MARKER: == sanity-lnet test 620: lnet_legacy2yaml: mixed legacy and alternative route formats ========================================================== 09:57:01 (1789135021) [ 3025.980488] Lustre: DEBUG MARKER: == sanity-lnet test 621: lnet_legacy2yaml: alternative format with invalid hopcount ========================================================== 09:57:10 (1789135030) [ 3033.456561] Lustre: DEBUG MARKER: == sanity-lnet test 622: lnet_legacy2yaml: alternative format with default-hop flag ========================================================== 09:57:18 (1789135038) [ 3043.437705] Lustre: DEBUG MARKER: == sanity-lnet test 623: lnet_legacy2yaml: basic networks conversion ========================================================== 09:57:27 (1789135047) [ 3053.195938] Lustre: DEBUG MARKER: == sanity-lnet test 624: lnet_legacy2yaml: networks with interfaces ========================================================== 09:57:37 (1789135057) [ 3062.041204] Lustre: DEBUG MARKER: == sanity-lnet test 625: lnet_legacy2yaml: multiple networks ========================================================== 09:57:47 (1789135067) [ 3073.238288] Lustre: DEBUG MARKER: == sanity-lnet test 626: lnet_legacy2yaml: mixed networks, ip2nets, and routes ========================================================== 09:57:57 (1789135077) [ 3082.258351] Lustre: DEBUG MARKER: == sanity-lnet test 627: lnet_legacy2yaml: invalid networks format ========================================================== 09:58:07 (1789135087) [ 3091.051746] Lustre: DEBUG MARKER: == sanity-lnet test 628: lnet_legacy2yaml: missing delimiter error ========================================================== 09:58:15 (1789135095) [ 3100.618628] Lustre: DEBUG MARKER: == sanity-lnet test 629: lnet_legacy2yaml: multiple gateways error ========================================================== 09:58:24 (1789135104) [ 3109.659147] Lustre: DEBUG MARKER: == sanity-lnet test complete, duration 2822 sec ========== 09:58:34 (1789135114) [ 3110.336151] LNet: 144399:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3110.349615] LNetError: 144399:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3110.373252] LNet: Removed LNI 192.168.201.55@tcp [ 3111.273509] Key type .llcrypt unregistered [ 3111.279600] Key type ._llcrypt unregistered [ 3129.373688] Key type ._llcrypt registered [ 3129.382184] Key type .llcrypt registered [ 3129.602395] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 3129.614873] alg: No test for adler32 (adler32-zlib) [ 3130.909397] Lustre: Lustre: Build Version: 2.17.57_131_gc13dec9 [ 3131.418171] LNet: Added LNI 192.168.201.55@tcp [8/256/0/180] [ 3133.193546] Key type lgssc registered [ 3134.778994] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3208.746395] Lustre: Mounted lustre-client - version 2.17.57_131_gc13dec9 [ 3215.060447] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3225.678293] Lustre: Unmounted lustre-client [ 3266.762506] Key type lgssc unregistered [ 3267.161307] LNet: 147377:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3267.169564] LNetError: 147377:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3267.182523] LNet: Removed LNI 192.168.201.55@tcp [ 3268.230176] Key type .llcrypt unregistered [ 3268.232521] Key type ._llcrypt unregistered