[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 421096738 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54d0-0x000f54df] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895288K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.002266] x2apic enabled [ 0.003000] Switched APIC routing to physical x2apic. [ 0.003014] kvm-guest: setup PV IPIs [ 0.005707] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.006018] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.007010] pid_max: default: 32768 minimum: 301 [ 0.008121] LSM: Security Framework initializing [ 0.009040] Yama: becoming mindful. [ 0.010029] SELinux: Initializing. [ 0.011045] *** VALIDATE selinux *** [ 0.015000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.019163] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.020108] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.021119] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.022105] *** VALIDATE tmpfs *** [ 0.024098] *** VALIDATE proc *** [ 0.025229] *** VALIDATE cgroup *** [ 0.026008] *** VALIDATE cgroup2 *** [ 0.027247] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.029097] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.030007] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.031027] Spectre V2 : User space: Vulnerable [ 0.032006] Speculative Store Bypass: Vulnerable [ 0.034579] debug: unmapping init [mem 0xffffffffbae59000-0xffffffffbae60fff] [ 0.036173] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.037692] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.038028] ... version: 2 [ 0.039012] ... bit width: 48 [ 0.040010] ... generic registers: 4 [ 0.041009] ... value mask: 0000ffffffffffff [ 0.042013] ... max period: 00007fffffffffff [ 0.043011] ... fixed-purpose events: 3 [ 0.044008] ... event mask: 000000070000000f [ 0.046227] rcu: Hierarchical SRCU implementation. [ 0.048357] smp: Bringing up secondary CPUs ... [ 0.049567] x86: Booting SMP configuration: [ 0.050022] .... node #0, CPUs: #1 #2 #3 [ 0.053103] smp: Brought up 1 node, 4 CPUs [ 0.055012] smpboot: Max logical packages: 1 [ 0.056016] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.175546] node 0 deferred pages initialised in 117ms [ 0.180135] devtmpfs: initialized [ 0.181220] x86/mm: Memory block size: 128MB [ 0.183653] gcov: version magic: 0x41383552 [ 0.186271] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.187066] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.188416] pinctrl core: initialized pinctrl subsystem [ 0.190266] [ 0.190876] ************************************************************* [ 0.194017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.197014] ** ** [ 0.199013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.205022] ** ** [ 0.207012] ** This means that this kernel is built to expose internal ** [ 0.210015] ** IOMMU data structures, which may compromise security on ** [ 0.212013] ** your system. ** [ 0.215015] ** ** [ 0.217013] ** If you see this message and you are not debugging the ** [ 0.219029] ** kernel, report this immediately to your vendor! ** [ 0.222015] ** ** [ 0.224016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.227016] ************************************************************* [ 0.230547] NET: Registered protocol family 16 [ 0.233429] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.236058] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.240071] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.244050] cpuidle: using governor menu [ 0.245680] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.248577] PCI: Using configuration type 1 for base access [ 0.250138] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.258236] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.261029] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.266073] cryptd: max_cpu_qlen set to 1000 [ 0.270361] ACPI: Added _OSI(Module Device) [ 0.272015] ACPI: Added _OSI(Processor Device) [ 0.273013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.275019] ACPI: Added _OSI(Processor Aggregator Device) [ 0.280470] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.287373] ACPI: Interpreter enabled [ 0.289065] ACPI: PM: (supports S0 S3 S4 S5) [ 0.290015] ACPI: Using IOAPIC for interrupt routing [ 0.292117] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.297391] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.308000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.310039] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.313023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.316115] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.321546] acpiphp: Slot [2] registered [ 0.323135] acpiphp: Slot [5] registered [ 0.325154] acpiphp: Slot [6] registered [ 0.326114] acpiphp: Slot [3] registered [ 0.328121] acpiphp: Slot [4] registered [ 0.329136] acpiphp: Slot [7] registered [ 0.330000] acpiphp: Slot [8] registered [ 0.330000] acpiphp: Slot [9] registered [ 0.331107] acpiphp: Slot [10] registered [ 0.333078] acpiphp: Slot [11] registered [ 0.334128] acpiphp: Slot [12] registered [ 0.336117] acpiphp: Slot [13] registered [ 0.338080] acpiphp: Slot [14] registered [ 0.339082] acpiphp: Slot [15] registered [ 0.341071] acpiphp: Slot [16] registered [ 0.342095] acpiphp: Slot [17] registered [ 0.344088] acpiphp: Slot [18] registered [ 0.345125] acpiphp: Slot [19] registered [ 0.347092] acpiphp: Slot [20] registered [ 0.348079] acpiphp: Slot [21] registered [ 0.350118] acpiphp: Slot [22] registered [ 0.351097] acpiphp: Slot [23] registered [ 0.353086] acpiphp: Slot [24] registered [ 0.355085] acpiphp: Slot [25] registered [ 0.357124] acpiphp: Slot [26] registered [ 0.358095] acpiphp: Slot [27] registered [ 0.360101] acpiphp: Slot [28] registered [ 0.362100] acpiphp: Slot [29] registered [ 0.363089] acpiphp: Slot [30] registered [ 0.365088] acpiphp: Slot [31] registered [ 0.367062] PCI host bridge to bus 0000:00 [ 0.368019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.371023] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.374025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.377025] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.379029] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.383014] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.385187] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.387909] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.391209] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.398437] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.401576] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.405021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.407013] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.411019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.416554] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.419074] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.421043] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.424769] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.428937] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.439015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.443015] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.449188] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.452722] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.461016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.473848] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.484349] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.489011] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.493000] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.506019] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.515594] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.518412] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.519349] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.522543] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.526252] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.531109] iommu: Default domain type: Passthrough [ 0.533304] SCSI subsystem initialized [ 0.535100] ACPI: bus type USB registered [ 0.536105] usbcore: registered new interface driver usbfs [ 0.538065] usbcore: registered new interface driver hub [ 0.539052] usbcore: registered new device driver usb [ 0.541114] pps_core: LinuxPPS API ver. 1 registered [ 0.542008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.545056] PTP clock support registered [ 0.547174] EDAC MC: Ver: 3.0.0 [ 0.549262] PCI: Using ACPI for IRQ routing [ 0.550872] NetLabel: Initializing [ 0.553022] NetLabel: domain hash size = 128 [ 0.555013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.557078] NetLabel: unlabeled traffic allowed by default [ 0.560127] vgaarb: loaded [ 0.561303] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.563017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.568308] clocksource: Switched to clocksource kvm-clock [ 0.665603] VFS: Disk quotas dquot_6.6.0 [ 0.666950] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.669665] *** VALIDATE ramfs *** [ 0.670741] *** VALIDATE hugetlbfs *** [ 0.672226] pnp: PnP ACPI init [ 0.674043] pnp: PnP ACPI: found 6 devices [ 0.696969] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.699540] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.701306] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.702723] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.704705] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.706868] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.709549] NET: Registered protocol family 2 [ 0.712041] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.716378] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.720474] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.725605] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.728749] TCP: Hash tables configured (established 65536 bind 65536) [ 0.731989] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.734820] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.736465] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.739284] NET: Registered protocol family 1 [ 0.741834] RPC: Registered named UNIX socket transport module. [ 0.744229] RPC: Registered udp transport module. [ 0.745554] RPC: Registered tcp transport module. [ 0.747235] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.749831] NET: Registered protocol family 44 [ 0.751471] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.753700] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.756022] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.758377] PCI: CLS 0 bytes, default 64 [ 0.760191] Unpacking initramfs... [ 2.145625] debug: unmapping init [mem 0xffff8e5bbcc64000-0xffff8e5bbffcffff] [ 2.149534] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.151934] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.154905] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.651252] Initialise system trusted keyrings [ 2.652785] Key type blacklist registered [ 2.654411] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.665885] zbud: loaded [ 2.669510] *** VALIDATE nfs *** [ 2.670445] *** VALIDATE nfs4 *** [ 2.671923] pstore: using deflate compression [ 2.675791] Platform Keyring initialized [ 2.768956] NET: Registered protocol family 38 [ 2.770809] Key type asymmetric registered [ 2.772403] Asymmetric key parser 'x509' registered [ 2.774306] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.777496] io scheduler mq-deadline registered [ 2.779232] io scheduler kyber registered [ 2.781033] io scheduler bfq registered [ 2.783044] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.786982] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.789589] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.792528] ACPI: Power Button [PWRF] [ 2.798298] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.803685] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.824647] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.855796] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.889335] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.894571] Non-volatile memory driver v1.3 [ 2.896766] Linux agpgart interface v0.103 [ 2.932292] virtio_blk virtio1: [vda] 136600 512-byte logical blocks (69.9 MB/66.7 MiB) [ 2.935853] vda: detected capacity change from 0 to 69939200 [ 2.949547] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.952849] vdb: detected capacity change from 0 to 1073741824 [ 2.959595] libphy: Fixed MDIO Bus: probed [ 2.965792] usbcore: registered new interface driver usbserial_generic [ 2.968193] usbserial: USB Serial support registered for generic [ 2.970528] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.974785] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.976703] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.978398] mousedev: PS/2 mouse device common for all mice [ 2.980211] rtc_cmos 00:05: RTC can wake from S4 [ 2.986479] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.990547] rtc_cmos 00:05: registered as rtc0 [ 2.992607] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.997222] intel_pstate: CPU model not supported [ 2.997724] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.003854] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.003907] hid: raw HID events driver (C) Jiri Kosina [ 3.008621] usbcore: registered new interface driver usbhid [ 3.010243] usbhid: USB HID core driver [ 3.011633] drop_monitor: Initializing network drop monitor service [ 3.013458] Initializing XFRM netlink socket [ 3.015334] NET: Registered protocol family 10 [ 3.018367] Segment Routing with IPv6 [ 3.019420] NET: Registered protocol family 17 [ 3.020883] mpls_gso: MPLS GSO support [ 3.026177] RAS: Correctable Errors collector initialized. [ 3.028097] AVX version of gcm_enc/dec engaged. [ 3.029668] AES CTR mode by8 optimization enabled [ 3.096782] sched_clock: Marking stable (3096755263, 0)->(3909960509, -813205246) [ 3.099722] registered taskstats version 1 [ 3.102227] Loading compiled-in X.509 certificates [ 3.104696] zswap: loaded using pool lzo/zbud [ 3.130370] Key type big_key registered [ 3.142108] Key type encrypted registered [ 3.143565] ima: No TPM chip found, activating TPM-bypass! [ 3.145178] ima: Allocated hash algorithm: sha1 [ 3.146907] ima: No architecture policies found [ 3.148761] evm: Initialising EVM extended attributes: [ 3.150622] evm: security.selinux [ 3.151784] evm: security.ima [ 3.152845] evm: security.capability [ 3.154044] evm: HMAC attrs: 0x1 [ 3.156256] rtc_cmos 00:05: setting system clock to 2026-06-12 03:30:57 UTC (1781235057) [ 3.162640] debug: unmapping init [mem 0xffffffffbbe03000-0xffffffffbbffffff] [ 3.165383] debug: unmapping init [mem 0xffffffffbab82000-0xffffffffbae58fff] [ 3.174161] Write protecting the kernel read-only data: 28672k [ 3.177704] debug: unmapping init [mem 0xffffffffb9203000-0xffffffffb93fffff] [ 3.180256] debug: unmapping init [mem 0xffffffffb9b14000-0xffffffffb9bfffff] [ 3.209905] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.219126] systemd[1]: Detected virtualization kvm. [ 3.220382] systemd[1]: Detected architecture x86-64. [ 3.221422] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.246684] systemd[1]: No hostname configured. [ 3.248271] systemd[1]: Set hostname to . [ 3.250288] random: systemd: uninitialized urandom read (16 bytes read) [ 3.252935] systemd[1]: Initializing machine ID from random generator. [ 3.296479] random: ln: uninitialized urandom read (6 bytes read) [ 3.368232] random: systemd: uninitialized urandom read (16 bytes read) [ 3.369818] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.373725] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.377841] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Timers. Starting Create Volatile Files and Directories... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Reached target Initrd Root Device. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.946703] device-mapper: uevent: version 1.0.3 [ 3.949134] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.637420] virtio_net virtio0 ens2: renamed from eth0 [ 4.664131] scsi host0: ata_piix [ 4.669193] scsi host1: ata_piix [ 4.671079] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 4.673833] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 8.366248] dracut-initqueue[581]: RTNETLINK answers: File exists [ 9.672968] random: crng init done [ 9.674236] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 9.975561] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.114797] printk: systemd: 25 output lines suppressed due to ratelimiting [ 11.371569] SELinux: Disabled at runtime. [ 11.431395] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.439349] systemd[1]: Detected virtualization kvm. [ 11.441233] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.903758] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.906837] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.910905] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.914749] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.918791] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.927075] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.936485] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. Mounting POSIX Message Queue File System... [ OK ] Created slice system-serial\x2dgetty.slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on Process Core Dump Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... [ 12.055154] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Stopped target Initrd File Systems. Mounting Huge Pages File System... Mounting Kernel Debug File System... [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 12.354737] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.693911] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.740393] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.824862] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.843331] EDAC sbridge: Ver: 1.1.2 [ 13.957918] Key type dns_resolver registered [ 14.239116] NFS: Registering the id_resolver key type [ 14.241400] Key type id_resolver registered [ 14.242830] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg646-client login: [ 42.619395] libcfs: loading out-of-tree module taints kernel. [ 42.693264] Key type ._llcrypt registered [ 42.700031] Key type .llcrypt registered [ 42.955202] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 42.964675] alg: No test for adler32 (adler32-zlib) [ 44.075913] Lustre: Lustre: Build Version: 2.17.53_82_geaee21b [ 44.479693] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 46.112343] Key type lgssc registered [ 46.913826] Lustre: Echo OBD driver; http://www.lustre.org/ [ 112.622789] Lustre: Mounted lustre-client [ 115.902660] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 123.813185] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing check_logdir /tmp/testlogs/ [ 126.037274] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing yml_node [ 127.941964] Lustre: DEBUG MARKER: Client: 2.17.53.82 [ 129.054663] Lustre: DEBUG MARKER: MDS: 2.17.53.82 [ 130.263236] Lustre: DEBUG MARKER: OSS: 2.17.53.82 [ 130.938798] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Thu Jun 11 23:33:04 EDT 2026 [ 138.210404] Lustre: lustre-OST0000-osc-ffff8e5c08e7e800: disconnect after 24s idle [ 138.592590] Lustre: Unmounted lustre-client [ 152.038062] Key type lgssc unregistered [ 152.189145] LNet: 8353:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 152.193713] LNetError: 8353:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 152.208677] LNet: Removed LNI 192.168.206.46@tcp [ 152.558876] Key type .llcrypt unregistered [ 152.560704] Key type ._llcrypt unregistered [ 163.234378] Key type ._llcrypt registered [ 163.235940] Key type .llcrypt registered [ 163.315853] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 163.325609] alg: No test for adler32 (adler32-zlib) [ 164.140561] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 164.175815] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 164.187803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show [ 165.849520] Lustre: DEBUG MARKER: excepting tests: 253 254 410 [ 165.893239] LNet: 9228:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 165.897391] LNetError: 9228:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 165.906489] LNet: Removed LNI 192.168.206.46@tcp [ 166.193858] Key type .llcrypt unregistered [ 166.195164] Key type ._llcrypt unregistered [ 168.402770] Key type ._llcrypt registered [ 168.404888] Key type .llcrypt registered [ 169.048770] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 23:33:42 (1781235222) [ 169.194642] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 169.202992] alg: No test for adler32 (adler32-zlib) [ 170.016728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 170.054554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 170.066812] LNet: 9644:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 170.080251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 170.106337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 170.116453] LNet: 9651:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 172.974784] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 23:33:46 (1781235226) [ 173.040271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 173.050993] LNet: 10226:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 173.060754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 173.095415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp [ 176.017466] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 23:33:49 (1781235229) [ 176.057885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 176.070716] LNet: 10807:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 176.084376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 176.132481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 2.2.2.2@o2ib [ 178.904696] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 23:33:52 (1781235232) [ 178.940845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 178.952159] LNet: 11390:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 178.965777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 179.005827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib [ 181.774179] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 23:33:55 (1781235235) [ 181.811991] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 181.837025] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 181.883666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp [ 182.081505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp [ 184.948381] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 23:33:58 (1781235238) [ 185.007866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 185.021143] LNet: 12566:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 185.024583] LNet: 12566:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 185.039491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 185.078764] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 185.265450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 185.445485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib [ 188.345120] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 23:34:02 (1781235242) [ 188.421541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 188.440912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 188.476490] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi,[6-12/4]@bxi3f [ 191.300244] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 23:34:05 (1781235245) [ 191.336290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 191.359404] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 191.400394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp [ 191.418555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 191.579951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib [ 191.598813] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 191.779523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp [ 191.796719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 191.956103] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib [ 191.973045] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 192.136505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib [ 192.155327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 192.307278] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni [ 192.323584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 192.483412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni [ 192.501562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 192.658746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi [ 192.672731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 192.829620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi [ 192.844485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 193.000698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f [ 193.017985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 193.189616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f --nid [8-12]@bxi3f [ 193.209151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 193.385839] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi,[3-7]@bxi3f [ 193.412088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 193.585781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 193.607956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 196.562341] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 23:34:10 (1781235250) [ 196.601730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 196.614724] LNet: 14422:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 196.617901] LNet: 14422:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 196.631882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 196.672836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp [ 196.689719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp [ 199.597664] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 23:34:13 (1781235253) [ 199.655261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 199.675884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 199.711650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 199.727715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 202.673959] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 23:34:16 (1781235256) [ 202.725985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 202.751353] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 202.803505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp [ 202.825459] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp [ 205.667039] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 23:34:19 (1781235259) [ 205.700055] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 205.719430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 205.769417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 205.788554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 208.932580] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 23:34:22 (1781235262) [ 208.992352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 209.031788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 209.085692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp [ 209.104590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp [ 212.294272] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 23:34:25 (1781235265) [ 212.358138] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 212.379083] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 212.418848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 212.445817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 215.783051] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 23:34:29 (1781235269) [ 215.835707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 215.855334] LNet: 17938:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 215.859032] LNet: 17938:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 215.871322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 215.929716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 216.115551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 219.233799] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 23:34:32 (1781235272) [ 219.284542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 219.316141] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 219.425665] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 219.561476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 222.597923] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 23:34:36 (1781235276) [ 222.640302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 222.665747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 222.741946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 222.848618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 225.920476] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 23:34:39 (1781235279) [ 226.050806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.075554] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.307745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 226.457182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 229.569376] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 23:34:43 (1781235283) [ 229.663735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 229.682036] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 229.714526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 229.739543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 233.091647] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 23:34:46 (1781235286) [ 233.150643] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 233.185281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 233.228559] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 233.267415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 236.753138] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 23:34:50 (1781235290) [ 236.800498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 236.828547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 236.883607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 19@gni [ 239.908374] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 23:34:53 (1781235293) [ 239.972757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 239.993511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 240.036917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib [ 242.907300] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 23:34:56 (1781235296) [ 242.964443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 242.986897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 243.026711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni [ 243.234488] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni [ 246.091709] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 23:34:59 (1781235299) [ 246.130456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 246.149648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 246.211069] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni [ 246.232232] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 22@gni --nid 26@gni [ 249.317857] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 23:35:02 (1781235302) [ 249.408265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 249.421156] LNet: 23963:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 249.426121] LNet: 23963:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 9 previous similar messages [ 249.443563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 249.492631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni [ 249.514416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni [ 252.534978] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 23:35:06 (1781235306) [ 252.627118] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 252.652591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 252.691827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni [ 252.721326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp [ 256.144933] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 23:35:09 (1781235309) [ 256.198128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 256.227676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 256.269964] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 256.291911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 259.463880] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 23:35:13 (1781235313) [ 259.511915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 259.535284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 259.567959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim [ 259.600187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp [ 259.613246] LNetError: 25732:0:(peer.c:2175:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 259.632858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --force [ 259.650685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show --nid 1.1.1.1@tcp [ 262.668929] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 23:35:16 (1781235316) [ 262.705539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 262.732626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 262.771042] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.k9XE2JF6XJ [ 262.809358] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.k9XE2JF6XJ [ 262.833226] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.k9XE2JF6XJ [ 265.767238] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 23:35:19 (1781235319) [ 265.889525] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 265.908951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 265.939270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp [ 265.959574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp [ 265.984688] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer list [ 268.701489] Lustre: DEBUG MARKER: == sanity-lnet test 50: Enable/disable routing via yaml import ========================================================== 23:35:22 (1781235322) [ 268.766920] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 268.791481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 268.831181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 269.930324] LNet: Message forwarding will be enabled in 2s [ 272.992199] LNet: Message forwarding enabled [ 273.168134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 273.181866] LNet: Message forwarding will stop in 2s to 6s [ 277.404671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 277.780066] LNet: Message forwarding disabled [ 278.453658] LNet: Message forwarding will be enabled in 2s [ 281.889037] LNet: Message forwarding enabled [ 282.157409] Lustre: DEBUG MARKER: == sanity-lnet test 51: Set numa range via yaml import === 23:35:35 (1781235335) [ 282.198035] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 282.829825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 282.876356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-51.yaml [ 286.196707] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 23:35:39 (1781235339) [ 286.244767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 286.268531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 286.292400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid foobar [ 286.312298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid foobar [ 286.327794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@o2ib [ 286.345391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --nid 1.1.1.1@tcp [ 286.363778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --nid 1.1.1.1@tcp [ 286.387840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add [ 286.416945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 286.455431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp [ 286.484262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@foo [ 286.509405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@tcp [ 286.533493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@o2ib [ 286.547099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@gni [ 286.561529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni [ 286.581206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni [ 286.602239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp [ 286.621096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp [ 286.643263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp [ 286.661356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 286.683866] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp --gateway 1@gni [ 286.702500] LNetError: 28798:0:(router.c:624:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni [ 289.909367] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 23:35:43 (1781235343) [ 289.951628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 289.978723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 290.032137] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 293.099506] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 23:35:46 (1781235346) [ 293.999752] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_100 Need at least 2 remote nodes found oleg646-server [ 294.881822] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 23:35:48 (1781235348) [ 294.963666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 294.989282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 295.016814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 295.034925] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 295.945868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp1 --gateway 192.168.206.[47-51]@tcp [ 295.963574] LNet: 30513:0:(router.c:728:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 295.990158] LNetError: 30303:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.206.47@tcp is being used as a gateway but routing feature is not turned on [ 295.997638] LNetError: 30303:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) route to tcp1 through 192.168.206.47@tcp has gone from up to down [ 296.999089] LNetError: 30303:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.206.51@tcp is being used as a gateway but routing feature is not turned on [ 297.007365] LNetError: 30303:0:(router.c:393:lnet_router_discovery_ping_reply()) Skipped 3 previous similar messages [ 298.023553] LNetError: 30303:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.206.47@tcp is being used as a gateway but routing feature is not turned on [ 298.037621] LNetError: 30303:0:(router.c:393:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message [ 299.040351] LNetError: 30309:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) route to tcp1 through 192.168.206.49@tcp has gone from up to down [ 299.051645] LNetError: 30309:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) Skipped 1 previous similar message [ 299.803384] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 23:35:53 (1781235353) [ 300.073288] LNetError: 30303:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.206.51@tcp is being used as a gateway but routing feature is not turned on [ 300.084794] LNetError: 30303:0:(router.c:393:lnet_router_discovery_ping_reply()) Skipped 3 previous similar messages [ 300.750314] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_102 Need at least 2 remote nodes found oleg646-server [ 301.769413] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 23:35:55 (1781235355) [ 301.827573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 302.112530] LNetError: 30302:0:(router.c:1262:lnet_check_routers()) Failed to discover router 192.168.206.47@tcp [ 302.881125] LNetError: 31441:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 302.892749] LNet: Removed LNI 192.168.206.46@tcp [ 302.908567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 302.943559] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 302.958600] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 303.295911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp103 --gateway 192.168.206.[47-52/2]@tcp [ 303.311708] LNet: 31565:0:(router.c:728:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 303.319631] LNet: 31565:0:(router.c:728:lnet_add_route()) Skipped 4 previous similar messages [ 303.336638] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route del --net tcp103 --gateway 192.168.206.[47-52/2]@tcp [ 303.339463] LNetError: 31452:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) route to tcp103 through 192.168.206.47@tcp has gone from up to down [ 303.351387] LNetError: 31452:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) Skipped 2 previous similar messages [ 306.403319] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 23:36:00 (1781235360) [ 306.486515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 306.499789] LNetError: 32157:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 306.519145] LNet: Removed LNI 192.168.206.46@tcp [ 306.533439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 306.577615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking -1 [ 306.620441] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 306.641410] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 306.675179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 306.710538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 306.730324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 306.759432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 0 [ 306.808280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 306.837960] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 306.878503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 306.914319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 306.931896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 306.966577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 1 [ 307.000508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 307.017982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 307.048416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 307.085985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 307.110238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 307.141884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 2 [ 307.182682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 307.204327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 307.245468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 307.289814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 307.310666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 307.339464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 307.383880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 307.412485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 307.456355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 307.502710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 307.521620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 307.552747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 4 [ 307.581574] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 307.605392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 307.649709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 310.611582] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 23:36:04 (1781235364) [ 311.414316] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_105 Need at least 2 remote nodes found oleg646-server [ 312.312512] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 23:36:05 (1781235365) [ 313.122245] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_106 Need at least 2 remote nodes found oleg646-server [ 313.872542] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 23:36:07 (1781235367) [ 313.925271] LNet: 33552:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 313.931939] LNet: 33552:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 23 previous similar messages [ 314.231247] Key type .llcrypt unregistered [ 314.232620] Key type ._llcrypt unregistered [ 316.822150] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 316.888387] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 316.901427] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 316.903424] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 319.140995] Key type ._llcrypt registered [ 319.142916] Key type .llcrypt registered [ 319.236702] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 319.244131] alg: No test for adler32 (adler32-zlib) [ 320.067055] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 320.095409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 320.110184] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 320.457468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 320.470197] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 321.433318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 321.446550] LNet: Removed LNI 10.1.2.3@tcp [ 321.497204] LNet: 34275:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 321.501421] LNetError: 34275:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 321.513114] LNet: Removed LNI 192.168.206.46@tcp [ 321.808350] Key type .llcrypt unregistered [ 321.809741] Key type ._llcrypt unregistered [ 326.545378] Key type ._llcrypt registered [ 326.546772] Key type .llcrypt registered [ 327.105390] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 23:36:20 (1781235380) [ 327.333457] Key type .llcrypt unregistered [ 327.336126] Key type ._llcrypt unregistered [ 329.452829] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 329.499268] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 329.502818] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 329.506902] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 331.743110] Key type ._llcrypt registered [ 331.744462] Key type .llcrypt registered [ 331.820256] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 331.827383] alg: No test for adler32 (adler32-zlib) [ 332.658774] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 332.688480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 332.705823] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 333.022936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 333.033331] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 334.029185] LNet: 35767:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 334.031866] LNetError: 35767:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 334.034594] LNet: Removed LNI 192.168.206.46@tcp [ 334.043780] LNet: Removed LNI 10.1.2.3@tcp [ 334.308524] Key type .llcrypt unregistered [ 334.310400] Key type ._llcrypt unregistered [ 338.758918] Key type ._llcrypt registered [ 338.761217] Key type .llcrypt registered [ 339.332542] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 23:36:33 (1781235393) [ 339.542513] Key type .llcrypt unregistered [ 339.544246] Key type ._llcrypt unregistered [ 342.071945] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 342.152404] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 342.156830] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 342.159032] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 344.419233] Key type ._llcrypt registered [ 344.420245] Key type .llcrypt registered [ 344.502064] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 344.511448] alg: No test for adler32 (adler32-zlib) [ 345.334218] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 345.364613] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 345.380653] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 346.364712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 346.375129] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 347.422097] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 347.434394] LNet: Removed LNI 10.1.2.3@tcp [ 347.445412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg:0 [ 347.477638] LNet: Removed LNI 10.1.2.31@tcp [ 347.491480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 347.507265] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 347.918950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 347.936400] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 348.367546] LNet: 37685:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 348.371443] LNetError: 37685:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 348.375740] LNet: Removed LNI 10.1.2.3@tcp [ 348.381641] LNet: Removed LNI 10.1.2.31@tcp [ 348.716359] Key type .llcrypt unregistered [ 348.717420] Key type ._llcrypt unregistered [ 354.217136] Key type ._llcrypt registered [ 354.218648] Key type .llcrypt registered [ 355.167977] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 23:36:48 (1781235408) [ 355.372189] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 355.382515] alg: No test for adler32 (adler32-zlib) [ 356.219984] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 356.259620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 356.281500] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 356.781901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 356.794828] LNet: 38636:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 356.798625] LNetError: 38636:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 356.807507] LNet: Removed LNI 192.168.206.46@tcp [ 356.822160] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 356.865403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --nid 192.168.206.46@tcp [ 356.884871] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 357.368194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --nid 192.168.206.46@tcp [ 357.399989] LNet: Removed LNI 192.168.206.46@tcp [ 357.460132] LNet: 38771:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 357.853663] Key type .llcrypt unregistered [ 357.856873] Key type ._llcrypt unregistered [ 362.443671] Key type ._llcrypt registered [ 362.446591] Key type .llcrypt registered [ 362.958831] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 23:36:56 (1781235416) [ 363.654519] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_111 Need at least 2 remote nodes found oleg646-server [ 364.258981] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 23:36:58 (1781235418) [ 364.438443] Key type .llcrypt unregistered [ 364.440057] Key type ._llcrypt unregistered [ 366.608855] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 366.695521] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 366.697518] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 366.703594] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 368.942577] Key type ._llcrypt registered [ 368.944475] Key type .llcrypt registered [ 369.029739] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 369.036669] alg: No test for adler32 (adler32-zlib) [ 369.888436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 369.922940] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 370.329629] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180] [ 371.363739] LNet: 40608:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 371.367948] LNetError: 40608:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 371.371821] LNet: Removed LNI 192.168.206.46@tcp1 [ 371.382700] LNet: Removed LNI 10.1.2.3@tcp2 [ 371.678791] Key type .llcrypt unregistered [ 371.681801] Key type ._llcrypt unregistered [ 376.241205] Key type ._llcrypt registered [ 376.242792] Key type .llcrypt registered [ 376.837724] Lustre: DEBUG MARKER: == sanity-lnet test 150: Check import of ip2nets yaml sequence ========================================================== 23:37:10 (1781235430) [ 376.995946] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 377.002845] alg: No test for adler32 (adler32-zlib) [ 377.810021] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 377.835348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 377.847194] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 378.214476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 378.223719] LNet: 41575:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 378.227569] LNetError: 41575:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 378.235923] LNet: Removed LNI 192.168.206.46@tcp [ 378.247676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 378.277935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet.yaml [ 378.292077] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 381.314629] Lustre: DEBUG MARKER: == sanity-lnet test 151: Check correct application of multiple ip2nets rules ========================================================== 23:37:15 (1781235435) [ 381.410374] LNet: 42165:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 381.414493] LNetError: 42165:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 381.423750] LNet: Removed LNI 192.168.206.46@tcp [ 381.707900] Key type .llcrypt unregistered [ 381.710163] Key type ._llcrypt unregistered [ 384.062637] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 384.118126] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 384.121109] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 384.123438] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 386.443626] Key type ._llcrypt registered [ 386.445194] Key type .llcrypt registered [ 386.560089] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 386.568199] alg: No test for adler32 (adler32-zlib) [ 387.427811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 387.479560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 387.500936] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 387.875470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 387.894708] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 388.926497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 388.945356] LNet: 42908:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 388.955961] LNetError: 42908:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 388.961252] LNet: Removed LNI 192.168.206.46@tcp [ 388.969792] LNet: Removed LNI 10.1.2.3@tcp [ 388.983285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 389.022293] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 389.037528] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 389.042116] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 389.246237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 389.257482] LNet: Removed LNI 192.168.206.46@tcp [ 389.264651] LNet: Removed LNI 10.1.2.3@tcp [ 389.280580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 389.321796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 389.346869] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 389.349922] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 389.571543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 389.588864] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 390.093158] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 390.103940] LNet: 43097:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 390.107786] LNet: 43097:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 390.114294] LNetError: 43097:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 390.117583] LNetError: 43097:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 390.120848] LNet: Removed LNI 192.168.206.46@tcp [ 390.122241] LNet: Removed LNI 10.1.2.3@tcp [ 390.136370] LNet: Removed LNI 192.168.206.46@tcp2 [ 390.149817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 390.183836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 390.199607] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 390.202634] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 390.207695] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 393.310920] Lustre: DEBUG MARKER: == sanity-lnet test 152: Check import of ip2nets with wildcard IP patterns ========================================================== 23:37:27 (1781235447) [ 393.368257] LNet: 43689:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 393.371986] LNetError: 43689:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 393.375654] LNetError: 43689:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 393.379571] LNet: Removed LNI 192.168.206.46@tcp [ 393.383238] LNet: Removed LNI 10.1.2.3@tcp [ 393.394770] LNet: Removed LNI 192.168.206.46@tcp2 [ 393.675777] Key type .llcrypt unregistered [ 393.680110] Key type ._llcrypt unregistered [ 396.384741] Key type ._llcrypt registered [ 396.387573] Key type .llcrypt registered [ 396.499885] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 396.514393] alg: No test for adler32 (adler32-zlib) [ 397.332247] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 397.358092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 397.372388] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 397.770606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 397.780400] LNet: 44086:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 397.783240] LNetError: 44086:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 397.795386] LNet: Removed LNI 192.168.206.46@tcp [ 397.804132] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 397.836830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 397.850946] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 398.045464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 398.067272] LNet: Removed LNI 192.168.206.46@tcp [ 398.080634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 398.115150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 398.128802] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 398.333958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 398.346301] LNet: 44146:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 398.349066] LNet: 44146:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 398.353202] LNetError: 44146:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 398.358177] LNetError: 44146:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 398.371407] LNet: Removed LNI 192.168.206.46@tcp [ 398.381870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 398.412280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 398.425614] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 398.625197] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 398.671438] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 398.674533] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 398.676882] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 400.868101] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 400.878337] LNet: 44226:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 400.881977] LNetError: 44226:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 400.891541] LNet: Removed LNI 192.168.206.46@tcp [ 400.901542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 400.941250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 400.955732] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 404.030597] Lustre: DEBUG MARKER: == sanity-lnet test 153: Check import of ip2nets with IP address ranges ========================================================== 23:37:37 (1781235457) [ 404.125866] LNet: 44820:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 404.131151] LNetError: 44820:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 404.144743] LNet: Removed LNI 192.168.206.46@tcp [ 404.446839] Key type .llcrypt unregistered [ 404.448159] Key type ._llcrypt unregistered [ 406.985771] Key type ._llcrypt registered [ 406.988071] Key type .llcrypt registered [ 407.127874] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 407.138264] alg: No test for adler32 (adler32-zlib) [ 407.961304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 407.992979] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 408.007631] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 408.420762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 408.430400] LNet: 45220:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 408.434926] LNetError: 45220:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 408.447294] LNet: Removed LNI 192.168.206.46@tcp [ 408.458278] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 408.493704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-153.yaml [ 408.507784] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 411.485267] Lustre: DEBUG MARKER: == sanity-lnet test 154: Check import of ip2nets with stepped IP address ranges ========================================================== 23:37:45 (1781235465) [ 411.563593] LNet: 45808:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 411.569458] LNetError: 45808:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 411.581276] LNet: Removed LNI 192.168.206.46@tcp [ 411.897769] Key type .llcrypt unregistered [ 411.903088] Key type ._llcrypt unregistered [ 414.536878] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 414.601984] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 414.606591] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 414.609365] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 416.963380] Key type ._llcrypt registered [ 416.965847] Key type .llcrypt registered [ 417.082614] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 417.093242] alg: No test for adler32 (adler32-zlib) [ 417.947481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 417.990645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 418.010418] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 418.504578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 418.515350] LNet: 46262:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 418.519392] LNetError: 46262:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 418.532653] LNet: Removed LNI 192.168.206.46@tcp [ 418.544132] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 418.581888] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-154.yaml [ 418.599278] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 421.861841] Lustre: DEBUG MARKER: == sanity-lnet test 155: Check ip2nets import failure with no matching rules ========================================================== 23:37:55 (1781235475) [ 421.918540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 421.934636] LNet: 46858:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 421.938961] LNetError: 46858:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 421.952676] LNet: Removed LNI 192.168.206.46@tcp [ 421.965653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 422.025735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-155.yaml [ 424.825994] Lustre: DEBUG MARKER: == sanity-lnet test 156: Check import of ip2nets with comma-separated IP ranges ========================================================== 23:37:58 (1781235478) [ 424.888612] LNet: 47437:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 425.178499] Key type .llcrypt unregistered [ 425.181670] Key type ._llcrypt unregistered [ 427.740228] Key type ._llcrypt registered [ 427.745166] Key type .llcrypt registered [ 427.870616] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 427.882970] alg: No test for adler32 (adler32-zlib) [ 428.715558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 428.744238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 428.758824] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 429.151979] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 429.168464] LNet: 47838:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 429.171804] LNetError: 47838:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 429.184458] LNet: Removed LNI 192.168.206.46@tcp [ 429.196704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 429.227550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-156.yaml [ 429.242069] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 432.194687] Lustre: DEBUG MARKER: == sanity-lnet test 157: Check import failure with malformed ip2nets YAML ========================================================== 23:38:05 (1781235485) [ 432.262808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 432.273452] LNet: 48428:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 432.279377] LNetError: 48428:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 432.288925] LNet: Removed LNI 192.168.206.46@tcp [ 432.307210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 432.342809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-157.yaml [ 435.430979] Lustre: DEBUG MARKER: == sanity-lnet test 158: Check import of ip2nets with single IPv6 address ========================================================== 23:38:09 (1781235489) [ 436.198585] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_158 Interface has no IPv6 [ 436.961504] Lustre: DEBUG MARKER: == sanity-lnet test 159: Check import of ip2nets with IPv6 /128 CIDR notation ========================================================== 23:38:10 (1781235490) [ 437.660803] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_159 Interface has no IPv6 [ 438.366451] Lustre: DEBUG MARKER: == sanity-lnet test 160: Check correct application of multiple IPv6 ip2nets rules ========================================================== 23:38:12 (1781235492) [ 439.083820] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_160 Interface has no IPv6 [ 439.849409] Lustre: DEBUG MARKER: == sanity-lnet test 161: Check import of ip2nets with various IPv6 prefix lengths ========================================================== 23:38:13 (1781235493) [ 440.599991] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_161 Interface has no IPv6 [ 441.365755] Lustre: DEBUG MARKER: == sanity-lnet test 162: Check ip2nets import failure with non-matching IPv6 ========================================================== 23:38:15 (1781235495) [ 442.112520] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_162 Interface has no IPv6 [ 442.940560] Lustre: DEBUG MARKER: == sanity-lnet test 170: Check CPTs and tunables parsing when importing ========================================================== 23:38:16 (1781235496) [ 443.018436] LNet: 50478:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 443.326649] Key type .llcrypt unregistered [ 443.328212] Key type ._llcrypt unregistered [ 445.642913] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 445.690595] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 445.693817] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 445.696242] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 447.930533] Key type ._llcrypt registered [ 447.932291] Key type .llcrypt registered [ 448.017629] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 448.025865] alg: No test for adler32 (adler32-zlib) [ 448.848127] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 448.888872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 448.902825] LNet: Added LNI 192.168.206.46@tcp [8/512/0/180] [ 448.908459] LNet: Added LNI 10.1.2.3@tcp [8/512/0/180] [ 448.911304] LNet: 50801:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 448.959274] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 448.969902] LNet: 50821:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 448.973767] LNetError: 50821:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 448.977806] LNet: Removed LNI 192.168.206.46@tcp [ 448.979613] LNetError: 50821:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 448.991497] LNet: Removed LNI 10.1.2.3@tcp [ 449.002315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 449.032688] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 449.045626] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 449.082889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 449.096640] LNet: 50845:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 449.100677] LNetError: 50845:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 449.111751] LNet: Removed LNI 192.168.206.46@tcp [ 449.122633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 449.160937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 449.175660] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 449.179161] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 449.181416] LNet: 50853:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 449.227593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 449.239797] LNet: 50871:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 449.243298] LNetError: 50871:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 449.247177] LNet: Removed LNI 192.168.206.46@tcp [ 449.249349] LNetError: 50871:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 449.263384] LNet: Removed LNI 10.1.2.3@tcp [ 449.275871] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 449.315444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 449.328281] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 449.333985] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 449.337237] LNet: 50879:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 449.416698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 449.427285] LNet: 50905:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 449.431797] LNetError: 50905:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 449.437941] LNet: Removed LNI 192.168.206.46@tcp [ 449.439411] LNetError: 50905:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 449.447463] LNet: Removed LNI 10.1.2.3@tcp [ 449.459369] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 449.491632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 449.510605] LNet: Added LNI 192.168.206.46@tcp [8/256/0/182] [ 449.514731] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 449.567162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 449.578920] LNet: 50933:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 449.583392] LNetError: 50933:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 449.586543] LNet: Removed LNI 192.168.206.46@tcp [ 449.589035] LNetError: 50933:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 449.598925] LNet: Removed LNI 10.1.2.3@tcp [ 449.611313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 449.646767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 449.669297] LNet: Added LNI 192.168.206.46@tcp [8/256/0/182] [ 449.674841] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 449.677607] LNet: 50941:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 449.726136] LNet: Removed LNI 192.168.206.46@tcp [ 449.734677] LNet: Removed LNI 10.1.2.3@tcp [ 450.039525] Key type .llcrypt unregistered [ 450.041167] Key type ._llcrypt unregistered [ 454.750586] Key type ._llcrypt registered [ 454.751716] Key type .llcrypt registered [ 455.494677] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 23:38:29 (1781235509) [ 455.703253] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 455.711739] alg: No test for adler32 (adler32-zlib) [ 456.532803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 456.560485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 456.578641] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 456.968552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --ip2net tcp 192.168.206.46 [ 456.979635] LNet: 51915:0:(config.c:1627:lnet_parse_ip2nets()) Kernel parsing of ip2nets is deprecated. Consider converting to LNet YAML configuration. [ 456.984231] LNet: Removed LNI 192.168.206.46@tcp [ 460.071078] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 23:38:33 (1781235513) [ 460.162150] LNet: 52484:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 460.167369] LNetError: 52484:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 460.177606] LNet: Removed LNI 192.168.206.46@tcp [ 460.479413] Key type .llcrypt unregistered [ 460.483610] Key type ._llcrypt unregistered [ 463.021882] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 463.127587] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 465.532293] Key type ._llcrypt registered [ 465.533993] Key type .llcrypt registered [ 465.612357] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 465.619865] alg: No test for adler32 (adler32-zlib) [ 466.622959] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 469.401515] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 23:38:43 (1781235523) [ 469.442758] LNet: 53390:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 469.447634] LNetError: 53390:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 469.456971] LNet: Removed LNI 10.1.2.3@tcp [ 469.723611] Key type .llcrypt unregistered [ 469.725040] Key type ._llcrypt unregistered [ 471.840158] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 471.942708] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 471.947068] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 471.949273] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 474.279508] Key type ._llcrypt registered [ 474.281030] Key type .llcrypt registered [ 474.357666] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 474.363940] alg: No test for adler32 (adler32-zlib) [ 475.197336] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 477.841974] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 23:38:51 (1781235531) [ 478.486287] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_202 kernel ip2nets deprecated in 2.17.53 [ 479.175674] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 23:38:52 (1781235532) [ 479.228102] LNet: 54571:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 479.231890] LNetError: 54571:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 479.241967] LNet: Removed LNI 10.1.2.3@tcp [ 479.529069] Key type .llcrypt unregistered [ 479.533282] Key type ._llcrypt unregistered [ 481.988355] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 482.102611] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 482.105631] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 482.108059] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 484.457875] Key type ._llcrypt registered [ 484.459302] Key type .llcrypt registered [ 484.537767] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 484.545138] alg: No test for adler32 (adler32-zlib) [ 485.368760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 485.411057] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 488.968538] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 23:39:02 (1781235542) [ 489.153121] LNet: 55805:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 489.155692] LNetError: 55805:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 489.164717] LNet: Removed LNI 10.1.2.3@tcp [ 489.435586] Key type .llcrypt unregistered [ 489.437217] Key type ._llcrypt unregistered [ 491.818828] Key type ._llcrypt registered [ 491.820093] Key type .llcrypt registered [ 491.916417] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 491.923912] alg: No test for adler32 (adler32-zlib) [ 492.737193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 492.750708] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 496.741912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 497.079498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_interrupt -r 1 [ 497.099922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_interrupt -r 1 [ 497.120820] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 497.137174] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 497.234537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 497.262333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 497.281463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 497.304559] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 497.319896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 497.337675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 497.354593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 497.370763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 497.383077] LNet: 56241:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 497.392234] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 497.410816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 497.425967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 497.442183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 497.458628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 497.474637] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 497.494363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 497.738462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_dropped -r 1 [ 497.757463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_dropped -r 1 [ 497.775901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 497.786877] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 497.891095] LNet: 1 local NIs in recovery (showing 1): 192.168.206.46@tcp [ 497.891339] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 497.915577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 497.930097] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 497.947589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 497.963278] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 497.980815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 497.998411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 498.016338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 498.026247] LNet: 56351:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 498.040239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 498.055639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 498.073691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 498.095463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 498.116598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 498.139142] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 498.165924] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 498.422731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_aborted -r 1 [ 498.441842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_aborted -r 1 [ 498.462332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 498.577453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 498.597299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 498.615946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 498.634761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 498.657167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 498.672241] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 498.688245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 498.705239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 498.721168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 498.739699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 498.760636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 498.779176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 498.795637] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 498.813189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 498.832753] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 499.034878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_no_route -r 1 [ 499.053787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_no_route -r 1 [ 499.072111] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 499.081365] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 499.083866] LNet: Skipped 1 previous similar message [ 499.192427] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 499.211268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 499.230653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 499.246501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 499.263047] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 499.276879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 499.290646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 499.307634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 499.316410] LNet: 56569:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 499.319635] LNet: 56569:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 499.328399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 499.342907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 499.358799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 499.374503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 499.391147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 499.409238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 499.426959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 499.638193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_timeout -r 1 [ 499.660710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_timeout -r 1 [ 499.684379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 499.785887] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 499.801902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 499.817945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 499.837447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 499.853272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 499.869274] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 499.887154] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 499.903399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 499.924248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 499.942786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 499.956881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 499.970326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 499.983501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 499.998068] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 500.013477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 500.220608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_error -r 1 [ 500.238936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_error -r 1 [ 500.255806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 500.347147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 500.362786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 500.376920] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 500.393149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 500.410401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 500.428935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 500.446737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 500.461239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 500.476355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 500.491860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 500.510109] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 500.524865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 500.538711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 500.555104] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 500.573185] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 500.692805] LNet: 56828:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 500.696923] LNetError: 56828:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 500.706657] LNet: Removed LNI 192.168.206.46@tcp [ 500.877639] Key type .llcrypt unregistered [ 500.879382] Key type ._llcrypt unregistered [ 507.957499] Key type ._llcrypt registered [ 507.958981] Key type .llcrypt registered [ 508.653423] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 23:39:22 (1781235562) [ 509.047583] Key type .llcrypt unregistered [ 509.051294] Key type ._llcrypt unregistered [ 511.931726] Key type ._llcrypt registered [ 511.934391] Key type .llcrypt registered [ 512.028601] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 512.035830] alg: No test for adler32 (adler32-zlib) [ 512.909787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 512.925628] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 517.211926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 521.791852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 521.812496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 521.823939] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 522.410935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_interrupt -r 1 [ 522.433185] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e local_interrupt -r 1 [ 522.458978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_interrupt -r 1 [ 522.482871] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e local_interrupt -r 1 [ 522.505986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e local_interrupt -r 1 [ 522.525813] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e local_interrupt -r 1 [ 522.545218] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e local_interrupt -r 1 [ 522.564630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e local_interrupt -r 1 [ 522.582935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 522.596696] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 522.600416] LNet: 1 local NIs in recovery (showing 1): 192.168.206.46@tcp [ 522.712777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 522.729594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 522.748765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 522.767450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 522.788416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 522.803236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 522.820947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 522.839975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 522.850265] LNet: 58291:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 522.862457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 522.879176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 522.895348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 522.915970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 522.933851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 522.951990] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 522.970328] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 523.198937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_dropped -r 1 [ 523.214705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e local_dropped -r 1 [ 523.231137] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_dropped -r 1 [ 523.249809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e local_dropped -r 1 [ 523.267822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e local_dropped -r 1 [ 523.285322] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e local_dropped -r 1 [ 523.302757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e local_dropped -r 1 [ 523.325536] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e local_dropped -r 1 [ 523.344957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 523.356426] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 523.360687] LNet: Skipped 2 previous similar messages [ 523.471407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 523.486255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 523.502622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 523.518305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 523.532325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 523.545934] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 523.564883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 523.578407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 523.589246] LNet: 58412:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 523.598720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 523.615654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 523.632672] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 523.650469] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 523.665951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 523.685473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 523.702422] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 523.951612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_aborted -r 1 [ 523.971572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e local_aborted -r 1 [ 523.993774] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_aborted -r 1 [ 524.020922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e local_aborted -r 1 [ 524.041183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e local_aborted -r 1 [ 524.058822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e local_aborted -r 1 [ 524.077992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e local_aborted -r 1 [ 524.095968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e local_aborted -r 1 [ 524.119773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 524.228966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 524.246530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 524.263444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 524.278344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 524.293338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 524.311074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 524.326304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 524.341713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 524.356547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 524.372677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 524.395244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 524.411276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 524.426472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 524.442374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 524.458791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 524.689519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_no_route -r 1 [ 524.710668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e local_no_route -r 1 [ 524.732731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_no_route -r 1 [ 524.752612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e local_no_route -r 1 [ 524.770700] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e local_no_route -r 1 [ 524.790060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e local_no_route -r 1 [ 524.812129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e local_no_route -r 1 [ 524.834877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e local_no_route -r 1 [ 524.853472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 524.864548] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 524.870277] LNet: Skipped 7 previous similar messages [ 524.983376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 525.000725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 525.015197] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 525.027907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 525.043875] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 525.058770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 525.078584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 525.094083] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 525.107511] LNet: 58654:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 525.111450] LNet: 58654:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 525.122560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 525.140626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 525.157376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 525.177533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 525.200907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 525.232412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 525.257525] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 525.462581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_timeout -r 1 [ 525.486989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e local_timeout -r 1 [ 525.506270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_timeout -r 1 [ 525.524151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e local_timeout -r 1 [ 525.546168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e local_timeout -r 1 [ 525.568799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e local_timeout -r 1 [ 525.592845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e local_timeout -r 1 [ 525.614824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e local_timeout -r 1 [ 525.639115] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 525.728635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 525.743600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 525.759519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 525.776841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 525.793898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 525.812035] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 525.832335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 525.848492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 525.863742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 525.880313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 525.896151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 525.911478] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 525.925745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 525.938262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 525.953645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 526.167136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_error -r 1 [ 526.187479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e local_error -r 1 [ 526.204630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_error -r 1 [ 526.224726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e local_error -r 1 [ 526.242933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e local_error -r 1 [ 526.264947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e local_error -r 1 [ 526.282403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e local_error -r 1 [ 526.301222] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e local_error -r 1 [ 526.321490] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 526.414722] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 526.429174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 526.443791] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 526.457595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 526.479377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 526.494951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 526.511080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 526.525992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 526.541671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 526.558731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 526.578043] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 526.596600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 526.616499] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 526.634225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 526.648928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 529.127429] LNet: 58972:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 529.132631] LNetError: 58972:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 530.144211] LNet: Removed LNI 192.168.206.46@tcp [ 530.146390] LNetError: 58972:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 530.159674] LNet: Removed LNI 192.168.206.46@tcp1 [ 530.361528] Key type .llcrypt unregistered [ 530.362586] Key type ._llcrypt unregistered [ 537.532658] Key type ._llcrypt registered [ 537.534182] Key type .llcrypt registered [ 538.257121] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 23:39:51 (1781235591) [ 538.677168] Key type .llcrypt unregistered [ 538.678467] Key type ._llcrypt unregistered [ 541.230067] Key type ._llcrypt registered [ 541.231853] Key type .llcrypt registered [ 541.342071] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 541.352662] alg: No test for adler32 (adler32-zlib) [ 542.186443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 542.202873] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 546.156986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 546.547748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e remote_dropped -r 1 [ 546.568489] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e remote_dropped -r 1 [ 546.594424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 546.607925] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 546.717122] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 546.735792] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 546.752120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 546.766436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 546.782326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 546.801076] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 546.818837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 546.837446] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 546.847150] LNet: 60219:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 546.857414] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 546.877620] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 546.894128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 546.910175] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 546.930165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 546.950088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 546.969059] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 547.202648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e remote_error -r 1 [ 547.221150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e remote_error -r 1 [ 547.239171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 547.251323] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 547.340456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 547.356292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 547.372674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 547.398400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 547.416450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 547.441894] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 547.459520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 547.479700] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 547.493955] LNet: 60328:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 547.504933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 547.525512] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 547.543180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 547.561436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 547.579496] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 547.598530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 547.615668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 547.856123] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e remote_timeout -r 1 [ 547.874063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e remote_timeout -r 1 [ 547.894266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 548.017736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 548.038869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 548.063629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 548.087266] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 548.105711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 548.123922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 548.146534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 548.186719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 548.218035] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 548.240903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 548.265624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 548.287280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 548.304560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 548.328052] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 548.348923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 548.485911] LNet: 60478:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 548.492997] LNetError: 60478:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 548.508360] LNet: Removed LNI 192.168.206.46@tcp [ 548.736401] Key type .llcrypt unregistered [ 548.738060] Key type ._llcrypt unregistered [ 555.063348] Key type ._llcrypt registered [ 555.065136] Key type .llcrypt registered [ 555.613951] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 23:40:09 (1781235609) [ 555.985405] Key type .llcrypt unregistered [ 555.987126] Key type ._llcrypt unregistered [ 558.264332] Key type ._llcrypt registered [ 558.265780] Key type .llcrypt registered [ 558.352159] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 558.360624] alg: No test for adler32 (adler32-zlib) [ 559.177669] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 559.191317] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 562.558758] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 565.829077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 565.844746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 565.859068] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 566.495245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e remote_dropped -r 1 [ 566.520906] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e remote_dropped -r 1 [ 566.539564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e remote_dropped -r 1 [ 566.557673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e remote_dropped -r 1 [ 566.575525] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e remote_dropped -r 1 [ 566.593236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e remote_dropped -r 1 [ 566.612540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e remote_dropped -r 1 [ 566.634956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e remote_dropped -r 1 [ 566.653153] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 566.663338] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 566.666221] LNet: 1 peer NIs in recovery (showing 1): 192.168.206.146@tcp [ 566.760624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 566.775776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 566.791194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 566.806788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 566.826248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 566.842995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 566.861062] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 566.876587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 566.885278] LNet: 61941:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 566.894472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 566.909895] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 566.925048] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 566.942795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 566.961311] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 566.978134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 566.995854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 567.016099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 567.218216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e remote_error -r 1 [ 567.237124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e remote_error -r 1 [ 567.254802] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e remote_error -r 1 [ 567.274232] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e remote_error -r 1 [ 567.297506] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e remote_error -r 1 [ 567.313762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e remote_error -r 1 [ 567.332229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e remote_error -r 1 [ 567.353292] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e remote_error -r 1 [ 567.372109] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 567.382081] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 567.385909] LNet: Skipped 2 previous similar messages [ 567.475479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 567.493789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 567.509162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 567.524122] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 567.540765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 567.555898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 567.570237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 567.586757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 567.597817] LNet: 62064:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 567.610227] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 567.629063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 567.646185] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 567.660251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 567.675588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 567.690189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 567.706598] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 567.722430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 567.945517] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e remote_timeout -r 1 [ 567.963975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e remote_timeout -r 1 [ 567.980504] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e remote_timeout -r 1 [ 567.997661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e remote_timeout -r 1 [ 568.016606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e remote_timeout -r 1 [ 568.046476] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e remote_timeout -r 1 [ 568.070468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e remote_timeout -r 1 [ 568.094613] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e remote_timeout -r 1 [ 568.116897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.146@tcp [ 568.221191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 568.236536] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 568.252976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 568.267451] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 568.282106] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 568.297182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 568.317471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 568.333666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 568.351873] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 568.368722] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 568.385961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 568.405212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 568.421926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 568.440593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 568.457754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 568.475850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 570.843085] LNet: 62264:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 570.847554] LNetError: 62264:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 570.850743] LNet: Removed LNI 192.168.206.46@tcp [ 570.859587] LNet: Removed LNI 192.168.206.46@tcp1 [ 571.042948] Key type .llcrypt unregistered [ 571.044572] Key type ._llcrypt unregistered [ 577.679366] Key type ._llcrypt registered [ 577.680600] Key type .llcrypt registered [ 578.284382] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 23:40:31 (1781235631) [ 579.031163] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_208 kernel ip2nets deprecated in 2.17.53 [ 579.705866] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 23:40:33 (1781235633) [ 580.049160] Key type .llcrypt unregistered [ 580.052318] Key type ._llcrypt unregistered [ 582.425442] Key type ._llcrypt registered [ 582.426435] Key type .llcrypt registered [ 582.510546] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 582.518457] alg: No test for adler32 (adler32-zlib) [ 583.338668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 583.354264] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 587.016193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 587.305056] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e network_timeout -r 1 [ 587.323505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e network_timeout -r 1 [ 587.340585] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 587.351671] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 587.437719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 587.452668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 587.465437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 587.480577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 587.489520] LNet: 1 local NIs in recovery (showing 1): 192.168.206.46@tcp [ 587.497923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 587.512872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 587.528370] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 587.545262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 587.554667] LNet: 63804:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 587.565400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 587.582856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 587.597348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 587.612301] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 587.627158] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 587.643850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 587.746077] LNet: 63843:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 587.748893] LNetError: 63843:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 587.758852] LNet: Removed LNI 192.168.206.46@tcp [ 587.923509] Key type .llcrypt unregistered [ 587.925342] Key type ._llcrypt unregistered [ 595.131582] Key type ._llcrypt registered [ 595.133564] Key type .llcrypt registered [ 595.229507] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 595.242662] alg: No test for adler32 (adler32-zlib) [ 596.078646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 596.100067] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 599.447079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 602.895919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 602.914070] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 602.925979] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 603.517578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e network_timeout -r 1 [ 603.536300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e network_timeout -r 1 [ 603.555501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e network_timeout -r 1 [ 603.573583] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e network_timeout -r 1 [ 603.592591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e network_timeout -r 1 [ 603.614221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e network_timeout -r 1 [ 603.632400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e network_timeout -r 1 [ 603.652246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e network_timeout -r 1 [ 603.675304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 603.686472] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 603.782605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 603.796844] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 603.812310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 603.827285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 603.848384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 603.864213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 603.878421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 603.894817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 603.903659] LNet: 64718:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 603.914460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 603.930403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 603.943728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 603.957514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 603.971264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 603.987468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 606.245470] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 606.368213] LNet: 1 local NIs in recovery (showing 1): 192.168.206.46@tcp [ 606.387848] LNet: 64792:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 606.393463] LNetError: 64792:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 606.398337] LNet: Removed LNI 192.168.206.46@tcp [ 606.404636] LNet: Removed LNI 192.168.206.46@tcp1 [ 606.559768] Key type .llcrypt unregistered [ 606.561451] Key type ._llcrypt unregistered [ 612.914621] Key type ._llcrypt registered [ 612.916010] Key type .llcrypt registered [ 613.480313] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 23:41:07 (1781235667) [ 613.640941] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 613.648996] alg: No test for adler32 (adler32-zlib) [ 614.469542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 614.501932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 614.518432] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 614.864125] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 614.874814] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 615.268377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 615.306907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 615.332564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 615.356693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 615.380301] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 615.401828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 615.423774] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.206.46@tcp [ 615.435223] LNetError: 65889:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.206.46@tcp to recovery [ 615.520330] LNet: 1 local NIs in recovery (showing 1): 192.168.206.46@tcp [ 615.520409] LNet: There was an unexpected network error while writing to 192.168.206.46: rc = -22 [ 617.568261] LNet: There was an unexpected network error while writing to 192.168.206.46: rc = -22 [ 618.658716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 621.665352] LNet: There was an unexpected network error while writing to 192.168.206.46: rc = -22 [ 621.855212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 621.874659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 621.907989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 621.919795] LNet: 65996:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 621.922668] LNetError: 65996:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 621.925310] LNet: Removed LNI 192.168.206.46@tcp [ 621.931355] LNet: Removed LNI 192.168.206.46@tcp1 [ 621.940372] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 621.968515] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 621.979965] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 622.305331] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 622.317791] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 622.680613] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 622.703278] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 622.737842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 622.760981] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 622.783262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 622.804228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 622.828480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 622.847487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.206.46@tcp [ 622.862525] LNetError: 66268:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.206.46@tcp to recovery [ 625.056310] LNet: There was an unexpected network error while writing to 192.168.206.46: rc = -22 [ 625.059766] LNet: Skipped 1 previous similar message [ 626.079963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 629.156535] LNet: There was an unexpected network error while writing to 192.168.206.46: rc = -22 [ 633.604182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 633.630189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 633.661811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 637.288504] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 23:41:30 (1781235690) [ 637.478845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 637.494910] LNet: 66984:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 637.506710] LNetError: 66984:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 637.512822] LNetError: 66984:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 637.519273] LNet: Removed LNI 192.168.206.46@tcp [ 637.529488] LNet: Removed LNI 192.168.206.46@tcp1 [ 637.546482] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 637.577439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 637.595108] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 638.028453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 638.042934] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 638.465531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 638.500546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 638.519853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 638.545320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 638.566634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 638.586952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 638.605977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.206.46@tcp --health 0 [ 638.624562] LNet: 1 peer NIs in recovery (showing 1): 192.168.206.46@tcp [ 638.660636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 639.649637] LNet: There was an unexpected network error while writing to 192.168.206.46: rc = -22 [ 639.652344] LNet: Skipped 1 previous similar message [ 654.230475] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 654.293084] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 654.310542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 654.326206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.206.46@tcp --health 500 [ 654.366086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 660.646155] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 660.681946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 660.693736] LNet: 67484:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 660.698396] LNetError: 67484:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 660.702125] LNetError: 67484:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 660.705968] LNet: Removed LNI 192.168.206.46@tcp [ 660.714788] LNet: Removed LNI 192.168.206.46@tcp1 [ 660.724662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 660.753948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 660.765561] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 661.082088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 661.093929] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 661.501880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 661.523958] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 661.553190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 661.568527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 661.584632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 661.601054] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 661.616804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 661.634029] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.206.46@tcp --health 0 [ 662.816488] LNet: There was an unexpected network error while writing to 192.168.206.46: rc = -22 [ 662.819780] LNet: Skipped 3 previous similar messages [ 663.810720] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 672.227959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 672.245160] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 672.260193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 675.083238] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 23:42:08 (1781235728) [ 675.289494] LNet: 68531:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 675.292879] LNetError: 68531:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 675.296173] LNetError: 68531:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 675.300133] LNet: Removed LNI 192.168.206.46@tcp [ 675.309617] LNet: Removed LNI 192.168.206.46@tcp1 [ 675.597793] Key type .llcrypt unregistered [ 675.600533] Key type ._llcrypt unregistered [ 677.798428] Key type ._llcrypt registered [ 677.800051] Key type .llcrypt registered [ 677.887387] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 677.899478] alg: No test for adler32 (adler32-zlib) [ 678.735971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 678.751134] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 682.946087] Lustre: DEBUG MARKER: Initial discovery [ 682.975255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.206.146@tcp [ 684.109506] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag [ 684.170531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 -e local_error [ 684.191980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.206.146@tcp [ 684.206997] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 684.220890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 684.246948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.206.146@tcp -r 1 -m GET -l 3 [ 684.270841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 192.168.206.146@tcp -r 1 -m GET -e local_error [ 684.291069] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d 192.168.206.146@tcp -r 1 -m PUT -l 6 [ 684.897186] LNet: 1 local NIs in recovery (showing 1): 192.168.206.46@tcp [ 684.966512] Lustre: DEBUG MARKER: Force oleg646-client.virtnet to discover 192.168.206.146@tcp (in background) [ 685.027752] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.206.146@tcp [ 685.759501] Lustre: DEBUG MARKER: Force oleg646-server to discover 192.168.206.46@tcp [ 686.875799] Lustre: DEBUG MARKER: Wait for 69285 [ 688.096373] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 688.730151] Lustre: DEBUG MARKER: Finished wait on 69285 [ 688.775239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay del -a [ 688.796273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 688.908819] LNet: 69741:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 688.913076] LNetError: 69741:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 688.924430] LNet: Removed LNI 192.168.206.46@tcp [ 689.126588] Key type .llcrypt unregistered [ 689.131226] Key type ._llcrypt unregistered [ 695.987174] Key type ._llcrypt registered [ 695.988557] Key type .llcrypt registered [ 696.618226] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 23:42:30 (1781235750) [ 696.835267] Key type .llcrypt unregistered [ 696.838070] Key type ._llcrypt unregistered [ 699.477646] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 699.516884] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 699.519106] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 699.521255] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 701.850690] Key type ._llcrypt registered [ 701.852571] Key type .llcrypt registered [ 701.945560] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 701.952656] alg: No test for adler32 (adler32-zlib) [ 702.790074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 702.825242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 702.843449] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 703.227264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 703.239300] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 707.462731] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 23:42:41 (1781235761) [ 707.584243] LNet: 71817:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 707.592940] LNetError: 71817:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 707.595944] LNet: Removed LNI 192.168.206.46@tcp [ 707.604879] LNet: Removed LNI 10.1.2.3@tcp [ 708.007160] Key type .llcrypt unregistered [ 708.009119] Key type ._llcrypt unregistered [ 711.253771] Key type ._llcrypt registered [ 711.255364] Key type .llcrypt registered [ 711.344806] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 711.352774] alg: No test for adler32 (adler32-zlib) [ 712.218313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 712.257481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 712.274044] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 712.662737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 712.677937] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 713.221214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 10.1.2.3@tcp 192.168.206.46@tcp [ 714.275809] LNetError: Unexpected error -22 connecting to 192.168.206.46@tcp at host 192.168.206.46:988 [ 716.826492] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 23:42:50 (1781235770) [ 716.879619] LNet: 72939:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 716.883486] LNetError: 72939:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 716.887645] LNet: Removed LNI 192.168.206.46@tcp [ 716.902272] LNet: Removed LNI 10.1.2.3@tcp [ 717.283873] Key type .llcrypt unregistered [ 717.290072] Key type ._llcrypt unregistered [ 720.319052] Key type ._llcrypt registered [ 720.320482] Key type .llcrypt registered [ 720.431684] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 720.439486] alg: No test for adler32 (adler32-zlib) [ 721.283268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 721.320925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 721.341568] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 721.757217] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 721.773159] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 722.310313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 192.168.206.46@tcp1 --nid 192.168.206.46@tcp2 [ 724.272681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.46@tcp1 192.168.206.46@tcp2 [ 724.284527] LNetError: 73783:0:(lib-move.c:2007:lnet_handle_find_routed_path()) Unable to send message from 192.168.206.46@tcp1 to 192.168.206.46@tcp2 - Route table may be misconfigured [ 724.298304] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.46@tcp2 192.168.206.46@tcp1 [ 727.104948] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 23:43:00 (1781235780) [ 727.186939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 727.198599] LNet: 74355:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 727.201524] LNetError: 74355:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 727.204847] LNet: Removed LNI 192.168.206.46@tcp1 [ 728.229744] LNet: Removed LNI 192.168.206.46@tcp2 [ 728.241918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 728.280562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 728.295827] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 728.643962] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 728.654729] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 729.042259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 729.063889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.46@tcp 192.168.206.46@tcp [ 729.081955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.46@tcp1 192.168.206.46@tcp1 [ 729.103794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.206.46@tcp -d 192.168.206.46@tcp -e network_timeout [ 729.121952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -e network_timeout [ 729.143219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -e network_timeout [ 729.161874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -e network_timeout [ 729.182480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 729.196973] LNet: There was an unexpected network error while writing to 192.168.206.46: rc = -22 [ 729.207442] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 729.257064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 729.305678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 729.313079] LNet: 1 local NIs in recovery (showing 1): 192.168.206.46@tcp [ 731.795367] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 23:43:05 (1781235785) [ 731.831923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 731.842938] LNet: 75211:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 731.846781] LNetError: 75211:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 731.849917] LNetError: 75211:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 731.852761] LNet: Removed LNI 192.168.206.46@tcp [ 732.902535] LNet: Removed LNI 192.168.206.46@tcp1 [ 732.916838] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 732.951693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 0@lo [ 733.055380] LNet: 75253:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 733.238158] Key type .llcrypt unregistered [ 733.239607] Key type ._llcrypt unregistered [ 737.435399] Key type ._llcrypt registered [ 737.436997] Key type .llcrypt registered [ 738.006440] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 23:43:11 (1781235791) [ 738.224153] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 738.235376] alg: No test for adler32 (adler32-zlib) [ 739.050659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 739.674112] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces [ 740.569927] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 23:43:14 (1781235794) [ 740.615817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 740.632960] LNet: 76343:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 740.654341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 740.696790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 740.715156] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 741.153591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 741.167364] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 741.648145] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 741.666468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp1 [ 741.684742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp1 [ 744.793598] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 23:43:18 (1781235798) [ 745.628725] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_220 Need at least 2 remote nodes found oleg646-server [ 746.304491] Lustre: DEBUG MARKER: == sanity-lnet test 221: Add routes w/DD disabled - check aliveness ========================================================== 23:43:20 (1781235800) [ 747.092306] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_221 Need at least 2 remote nodes found oleg646-server [ 747.833923] Lustre: DEBUG MARKER: == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 23:43:21 (1781235801) [ 748.648615] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_222 Need at least 2 remote nodes found oleg646-server [ 749.429085] Lustre: DEBUG MARKER: == sanity-lnet test 223: Check avoid_asym_router_failure=1 w/DD disabled ========================================================== 23:43:23 (1781235803) [ 750.290808] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_223 Need at least 2 remote nodes found oleg646-server [ 751.191277] Lustre: DEBUG MARKER: == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 23:43:24 (1781235804) [ 751.976774] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_224 Need at least 2 remote nodes found oleg646-server [ 752.680108] Lustre: DEBUG MARKER: == sanity-lnet test 225: Check avoid_asym_router_failure=0 w/DD disabled ========================================================== 23:43:26 (1781235806) [ 753.435398] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_225 Need at least 2 remote nodes found oleg646-server [ 754.163089] Lustre: DEBUG MARKER: == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 23:43:27 (1781235807) [ 754.970542] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_226 Need at least 3 remote nodes found oleg646-server [ 755.731282] Lustre: DEBUG MARKER: == sanity-lnet test 227: Check router peer health w/DD disabled ========================================================== 23:43:29 (1781235809) [ 756.610610] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_227 Need at least 3 remote nodes found oleg646-server [ 757.404881] Lustre: DEBUG MARKER: == sanity-lnet test 228: Routes should stay up when health is decremented ========================================================== 23:43:31 (1781235811) [ 758.185944] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_228 Need at least 3 remote nodes found oleg646-server [ 758.975798] Lustre: DEBUG MARKER: == sanity-lnet test 230: Test setting conns-per-peer ===== 23:43:32 (1781235812) [ 759.007904] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 759.018745] LNet: 80217:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 759.022090] LNetError: 80217:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 759.025633] LNet: Removed LNI 192.168.206.46@tcp [ 759.035961] LNet: Removed LNI 192.168.206.46@tcp1 [ 759.046155] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 759.079176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 759.095965] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 759.449663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 4 [ 759.503463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 759.590661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 759.602634] LNetError: 80370:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 759.607078] LNetError: 80370:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 759.621804] LNet: Removed LNI 192.168.206.46@tcp [ 759.637952] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 759.667404] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 5 [ 759.680689] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 760.034790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 761.192904] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 761.209502] LNet: 80524:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 761.218534] LNet: 80524:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 761.222988] LNetError: 80524:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 762.279704] LNet: Removed LNI 192.168.206.46@tcp [ 762.288814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 762.320492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 762.331989] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 762.656790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 6 [ 762.690645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 762.778555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 763.815728] LNet: Removed LNI 192.168.206.46@tcp [ 763.829870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 763.868283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 7 [ 763.884344] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 764.292652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 764.410086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 764.432992] LNet: 80817:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 764.436610] LNet: 80817:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 764.440941] LNetError: 80817:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 764.444888] LNetError: 80817:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 764.456661] LNet: Removed LNI 192.168.206.46@tcp [ 764.472293] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 764.508173] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 764.527927] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 764.895932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 8 [ 764.953343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 765.067500] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 766.119689] LNet: Removed LNI 192.168.206.46@tcp [ 766.138248] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 766.180484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 9 [ 766.194304] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 766.583188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 766.667881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 767.718616] LNet: Removed LNI 192.168.206.46@tcp [ 767.729907] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 767.759075] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 767.771305] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 768.146695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 10 [ 768.190469] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 768.304902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 769.385918] LNet: Removed LNI 192.168.206.46@tcp [ 769.404193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 769.446171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 11 [ 769.463238] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 769.848230] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 769.945090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 769.959185] LNet: 81401:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 769.963221] LNet: 81401:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 769.967919] LNetError: 81401:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 769.973392] LNetError: 81401:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 769.983604] LNet: Removed LNI 192.168.206.46@tcp [ 769.995569] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 770.026463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 770.039443] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 770.382094] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 12 [ 770.424448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 770.520692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 771.557804] LNet: Removed LNI 192.168.206.46@tcp [ 771.568864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 771.598953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 13 [ 771.616956] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 771.959174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 772.083599] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 773.160377] LNet: Removed LNI 192.168.206.46@tcp [ 773.169667] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 773.199902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 773.214464] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 773.625995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 14 [ 773.685889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 774.949795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 776.038626] LNet: Removed LNI 192.168.206.46@tcp [ 776.050984] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 776.079942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 15 [ 776.097844] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 776.555698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 776.715607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 777.767627] LNet: Removed LNI 192.168.206.46@tcp [ 777.789673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 777.815280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 777.830309] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 778.184158] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 128 [ 778.218134] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 778.229457] LNet: 82128:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 778.232149] LNet: 82128:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 4 previous similar messages [ 778.235645] LNetError: 82128:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 778.239212] LNetError: 82128:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 4 previous similar messages [ 778.246727] LNet: Removed LNI 192.168.206.46@tcp [ 778.257897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 778.287397] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 778.300695] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 778.647900] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer -1 [ 781.731828] Lustre: DEBUG MARKER: == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 23:43:55 (1781235835) [ 781.803124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 781.822459] LNet: Removed LNI 192.168.206.46@tcp [ 781.837343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 781.863896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 [ 781.876837] LNet: Added LNI 192.168.206.46@tcp231 [8/256/0/180] [ 782.399976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 782.423941] LNet: Removed LNI 192.168.206.46@tcp231 [ 782.437847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 782.478605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 782.494539] LNet: Added LNI 192.168.206.46@tcp231 [8/256/0/180] [ 782.497729] LNet: 82977:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 782.700659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 782.724464] LNet: Removed LNI 192.168.206.46@tcp231 [ 782.735264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=-1 [ 782.753588] LNet: Added LNI 192.168.206.46@tcp231 [8/256/0/180] [ 783.275898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 783.295708] LNet: Removed LNI 192.168.206.46@tcp231 [ 783.306327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 783.353767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 783.373966] LNet: Added LNI 192.168.206.46@tcp231 [8/256/0/0] [ 783.377065] LNet: 83129:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 783.599098] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 783.619985] LNet: Removed LNI 192.168.206.46@tcp231 [ 783.631527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=0 [ 783.646563] LNet: Added LNI 192.168.206.46@tcp231 [8/256/0/0] [ 784.211694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 784.229751] LNet: Removed LNI 192.168.206.46@tcp231 [ 784.245804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 784.286314] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 784.302521] LNet: Added LNI 192.168.206.46@tcp231 [8/256/0/60] [ 784.499272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 784.521693] LNet: Removed LNI 192.168.206.46@tcp231 [ 784.536455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=60 [ 784.551638] LNet: Added LNI 192.168.206.46@tcp231 [8/256/0/60] [ 785.042578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 785.068469] LNet: Removed LNI 192.168.206.46@tcp231 [ 785.076845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 785.113384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 785.128382] LNet: Added LNI 192.168.206.46@tcp231 [8/256/0/180] [ 785.132393] LNet: 83431:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 785.138800] LNet: 83431:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 785.330272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 785.351707] LNet: Removed LNI 192.168.206.46@tcp231 [ 785.363209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=180 [ 785.378853] LNet: Added LNI 192.168.206.46@tcp231 [8/256/0/180] [ 788.398993] Lustre: DEBUG MARKER: == sanity-lnet test 232: Test setting ToS value ========== 23:44:02 (1781235842) [ 788.438577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 788.459889] LNet: Removed LNI 192.168.206.46@tcp231 [ 788.471264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 788.501158] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 [ 788.515036] LNet: Added LNI 192.168.206.46@tcp232 [8/256/0/180] [ 788.829168] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 788.848254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 788.864938] LNet: Removed LNI 192.168.206.46@tcp232 [ 788.874455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos -1 [ 788.888271] LNet: Added LNI 192.168.206.46@tcp232 [8/256/0/180] [ 789.248070] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 789.271351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 789.286864] LNet: Removed LNI 192.168.206.46@tcp232 [ 789.298630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 104 [ 789.314275] LNet: Added LNI 192.168.206.46@tcp232 [8/256/0/180] [ 789.751407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 789.773693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 789.787717] LNet: Removed LNI 192.168.206.46@tcp232 [ 789.795565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 106 [ 789.810925] LNet: Added LNI 192.168.206.46@tcp232 [8/256/0/180] [ 790.146532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 790.163810] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 790.179652] LNet: Removed LNI 192.168.206.46@tcp232 [ 790.188175] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 300 [ 792.913518] Lustre: DEBUG MARKER: == sanity-lnet test 233: Check for successful resends ==== 23:44:06 (1781235846) [ 793.378219] Key type .llcrypt unregistered [ 793.381070] Key type ._llcrypt unregistered [ 795.696759] Key type ._llcrypt registered [ 795.698516] Key type .llcrypt registered [ 795.777934] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 795.786404] alg: No test for adler32 (adler32-zlib) [ 796.607028] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 796.621776] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 800.062253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 803.401453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 803.420653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 803.432130] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 804.071862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_interrupt -r 2 [ 804.090629] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e local_interrupt -r 2 [ 804.113319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_interrupt -r 2 [ 804.132590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e local_interrupt -r 2 [ 804.152843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e local_interrupt -r 2 [ 804.172244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e local_interrupt -r 2 [ 804.193543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e local_interrupt -r 2 [ 804.216167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e local_interrupt -r 2 [ 804.234724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.46@tcp 192.168.206.146@tcp [ 804.247852] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 804.252357] LNet: 1 local NIs in recovery (showing 1): 192.168.206.46@tcp [ 804.347538] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 804.364396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 804.379965] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 804.396825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 804.412919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 804.429363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 804.446667] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 804.462771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 804.472345] LNet: 85928:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 804.491661] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 804.517351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 804.535541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 804.551463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 804.567890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 804.584382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 804.822630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_dropped -r 2 [ 804.842258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e local_dropped -r 2 [ 804.863854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_dropped -r 2 [ 804.885152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e local_dropped -r 2 [ 804.907916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e local_dropped -r 2 [ 804.930878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e local_dropped -r 2 [ 804.953335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e local_dropped -r 2 [ 804.975436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e local_dropped -r 2 [ 804.999928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.46@tcp 192.168.206.146@tcp [ 805.014647] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 805.127969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 805.147520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 805.162967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 805.182202] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 805.202289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 805.220316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 805.235157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 805.258063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 805.278465] LNet: 86048:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 805.289367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 805.310670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 805.325784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 805.341770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 805.368827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 805.387444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 805.692682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_aborted -r 2 [ 805.713359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e local_aborted -r 2 [ 805.738385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_aborted -r 2 [ 805.761507] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e local_aborted -r 2 [ 805.784765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e local_aborted -r 2 [ 805.811280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e local_aborted -r 2 [ 805.833592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e local_aborted -r 2 [ 805.857618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e local_aborted -r 2 [ 805.878708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.46@tcp 192.168.206.146@tcp [ 805.992577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 806.010897] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 806.029872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 806.052796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 806.071751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 806.086857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 806.102167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 806.117968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 806.136209] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 806.155544] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 806.172650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 806.188335] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 806.205529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 806.227932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 806.437747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_no_route -r 2 [ 806.456822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e local_no_route -r 2 [ 806.478356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_no_route -r 2 [ 806.499417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e local_no_route -r 2 [ 806.516271] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e local_no_route -r 2 [ 806.534618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e local_no_route -r 2 [ 806.555225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e local_no_route -r 2 [ 806.578491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e local_no_route -r 2 [ 806.597668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.46@tcp 192.168.206.146@tcp [ 806.613450] LNet: There was an unexpected network error while writing to 192.168.206.146: rc = -22 [ 806.617322] LNet: Skipped 2 previous similar messages [ 806.721213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 806.737366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 806.753670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 806.769467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 806.785612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 806.801107] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 806.814518] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 806.831532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 806.841145] LNet: 86288:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 806.844843] LNet: 86288:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 806.855342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 806.871223] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 806.887729] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 806.903835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 806.920890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 806.941155] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 807.175560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e local_timeout -r 2 [ 807.203578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e local_timeout -r 2 [ 807.226874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e local_timeout -r 2 [ 807.247612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e local_timeout -r 2 [ 807.269415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e local_timeout -r 2 [ 807.288390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e local_timeout -r 2 [ 807.305972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e local_timeout -r 2 [ 807.326398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e local_timeout -r 2 [ 807.349044] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.46@tcp 192.168.206.146@tcp [ 807.459513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 807.475471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 807.491196] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 807.508650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 807.526740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 807.545023] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 807.564484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 807.583265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 807.604604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 807.620556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 807.636931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 807.653931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 807.670904] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 807.689919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 807.915927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp -m GET -e remote_dropped -r 2 [ 807.937565] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.146@tcp1 -m GET -e remote_dropped -r 2 [ 807.954727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp -m GET -e remote_dropped -r 2 [ 807.973801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp -d 192.168.206.46@tcp1 -m GET -e remote_dropped -r 2 [ 807.992531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp -m GET -e remote_dropped -r 2 [ 808.011149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.46@tcp1 -d 192.168.206.146@tcp1 -m GET -e remote_dropped -r 2 [ 808.030675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp -m GET -e remote_dropped -r 2 [ 808.049723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.46@tcp1 -d 192.168.206.46@tcp1 -m GET -e remote_dropped -r 2 [ 808.069572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.46@tcp 192.168.206.146@tcp [ 808.082000] LNet: 1 peer NIs in recovery (showing 1): 192.168.206.146@tcp [ 808.166225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 808.179589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 808.194713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 808.211780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 808.228456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 808.245864] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 808.260648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 808.282835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 808.298794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 808.315527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 808.334532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 808.352084] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 808.369758] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 808.387471] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 810.922645] LNet: 86602:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 810.925383] LNetError: 86602:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 810.928796] LNet: Removed LNI 192.168.206.46@tcp [ 810.941640] LNet: Removed LNI 192.168.206.46@tcp1 [ 811.141089] Key type .llcrypt unregistered [ 811.143812] Key type ._llcrypt unregistered [ 818.108115] Key type ._llcrypt registered [ 818.109479] Key type .llcrypt registered [ 818.755912] Lustre: DEBUG MARKER: == sanity-lnet test 234: Verify NI status changes with manual health value transitions ========================================================== 23:44:32 (1781235872) [ 818.966995] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 818.982037] alg: No test for adler32 (adler32-zlib) [ 819.812493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 819.841285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 819.857044] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 820.244425] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 0 [ 820.264617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.206.46@tcp [ 820.277658] LNetError: 87578:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.206.46@tcp to recovery [ 820.321642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --nid 192.168.206.46@tcp [ 820.377926] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 823.664334] Lustre: DEBUG MARKER: == sanity-lnet test 235: Verify NI status DOWN when health reaches 0 after failures ========================================================== 23:44:37 (1781235877) [ 823.774916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 823.788292] LNet: 88166:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 823.792916] LNetError: 88166:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 823.803986] LNet: Removed LNI 192.168.206.46@tcp [ 823.816804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 823.851512] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 823.869021] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 824.244107] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 824.257695] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 824.781348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 100 --nid 192.168.206.46@tcp [ 824.797280] LNetError: 88432:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.206.46@tcp to recovery [ 824.864439] LNet: 1 local NIs in recovery (showing 1): 192.168.206.46@tcp [ 824.865104] LNet: There was an unexpected network error while writing to 192.168.206.46: rc = -22 [ 825.932563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 900 --nid 192.168.206.46@tcp [ 830.094832] Lustre: DEBUG MARKER: == sanity-lnet test 236: Local NI state propagates to routers ========================================================== 23:44:43 (1781235883) [ 831.117129] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_236 Need at least 2 remote nodes found oleg646-server [ 832.077390] Lustre: DEBUG MARKER: == sanity-lnet test 237: Check MR forwarding feature ===== 23:44:45 (1781235885) [ 832.957810] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_237 Need at least 2 remote nodes found oleg646-server [ 833.856185] Lustre: DEBUG MARKER: == sanity-lnet test 238: Check MR forwarding feature skipped for non-MR source ========================================================== 23:44:47 (1781235887) [ 834.768587] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_238 Need at least 2 remote nodes found oleg646-server [ 835.770258] Lustre: DEBUG MARKER: == sanity-lnet test 241: Check setting LND timeout value via lnetctl updates tunables ========================================================== 23:44:49 (1781235889) [ 835.822840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 835.851138] LNet: 90046:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 835.860399] LNetError: 90046:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 835.864944] LNet: Removed LNI 192.168.206.46@tcp [ 835.890678] LNet: Removed LNI 192.168.206.46@tcp1 [ 835.910187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 835.945575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 835.962761] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 836.428578] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set -h [ 836.478061] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 51 [ 836.509777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 836.557353] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 0 [ 836.590785] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 836.620213] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 50 [ 839.997695] Lustre: DEBUG MARKER: == sanity-lnet test 245: Check read-only params do not appear in export --backup ========================================================== 23:44:53 (1781235893) [ 840.126579] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 840.139661] LNet: 90778:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 840.146350] LNetError: 90778:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 840.151962] LNetError: 90778:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 840.165712] LNet: Removed LNI 192.168.206.46@tcp [ 840.177436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 840.215180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 840.232793] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 843.434652] Lustre: DEBUG MARKER: == sanity-lnet test 250: test that linux routes are added ========================================================== 23:44:57 (1781235897) [ 843.539277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 843.552798] LNet: 91481:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 843.556987] LNetError: 91481:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 843.571942] LNet: Removed LNI 192.168.206.46@tcp [ 843.582687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 843.609432] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 843.621538] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 846.620461] Lustre: DEBUG MARKER: == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 23:45:00 (1781235900) [ 847.408791] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_251 Need kfi NETTYPE [ 848.132555] Lustre: DEBUG MARKER: == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 23:45:01 (1781235901) [ 848.330670] LNet: 92515:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 848.335221] LNetError: 92515:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 848.343970] LNet: Removed LNI 192.168.206.46@tcp [ 848.602719] Key type .llcrypt unregistered [ 848.604347] Key type ._llcrypt unregistered [ 851.020434] Key type ._llcrypt registered [ 851.023487] Key type .llcrypt registered [ 851.111371] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 851.122051] alg: No test for adler32 (adler32-zlib) [ 851.949251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 851.962092] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 855.460995] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 857.219385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --timeout 15 192.168.206.146@tcp [ 857.375381] LNet: 92930:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 857.380275] LNetError: 92930:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 857.394440] LNet: Removed LNI 192.168.206.46@tcp [ 857.578452] Key type .llcrypt unregistered [ 857.580108] Key type ._llcrypt unregistered [ 861.695980] Key type ._llcrypt registered [ 861.697575] Key type .llcrypt registered [ 862.211551] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253 [ 862.881753] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254 [ 863.494990] Lustre: DEBUG MARKER: == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 23:45:17 (1781235917) [ 864.064453] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_255 kernel ip2nets deprecated in 2.17.53 [ 864.733799] Lustre: DEBUG MARKER: == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 23:45:18 (1781235918) [ 865.478204] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_256 Need at least 2 remote nodes found oleg646-server [ 866.206281] Lustre: DEBUG MARKER: == sanity-lnet test 257: Test graceful router shutdown/startup ========================================================== 23:45:19 (1781235919) [ 866.976034] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_257 Need at least 3 remote nodes found oleg646-server [ 867.807640] Lustre: DEBUG MARKER: == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 23:45:21 (1781235921) [ 868.355739] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 868.398662] alg: No test for adler32 (adler32-zlib) [ 869.338808] Lustre: Lustre: Build Version: 2.17.53_82_geaee21b [ 869.496975] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 871.113911] Key type lgssc registered [ 871.662630] Lustre: Echo OBD driver; http://www.lustre.org/ [ 881.909364] Key type lgssc unregistered [ 882.057245] LNet: 96099:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 882.064256] LNetError: 96099:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 882.076906] LNet: Removed LNI 192.168.206.46@tcp [ 882.418676] Key type .llcrypt unregistered [ 882.420925] Key type ._llcrypt unregistered [ 889.130435] Key type ._llcrypt registered [ 889.132961] Key type .llcrypt registered [ 889.708861] Lustre: DEBUG MARKER: == sanity-lnet test 265: Import of invalid CPT should fail ========================================================== 23:45:43 (1781235943) [ 889.892243] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 889.899759] alg: No test for adler32 (adler32-zlib) [ 890.722161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 890.763759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 890.795986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 890.816788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 890.841985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 893.972369] Lustre: DEBUG MARKER: == sanity-lnet test 266: Validate CPT parsing in network sequence ========================================================== 23:45:47 (1781235947) [ 894.125477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 894.140611] LNet: 97512:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 894.152592] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 894.189603] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 894.212995] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 894.691355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 894.705327] LNet: 97656:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 894.710440] LNetError: 97656:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 894.720373] LNet: Removed LNI 192.168.206.46@tcp [ 894.733416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 894.767204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 894.782074] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 895.011286] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 895.026603] LNet: Removed LNI 192.168.206.46@tcp [ 895.038154] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 895.071020] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 895.089173] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 898.144946] Lustre: DEBUG MARKER: == sanity-lnet test 270: Test NID metadata =============== 23:45:51 (1781235951) [ 898.190070] LNet: 98268:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 898.194195] LNet: 98268:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 898.198426] LNetError: 98268:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 898.201353] LNetError: 98268:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 898.209917] LNet: Removed LNI 192.168.206.46@tcp [ 898.552669] Key type .llcrypt unregistered [ 898.554716] Key type ._llcrypt unregistered [ 901.010170] Key type ._llcrypt registered [ 901.012158] Key type .llcrypt registered [ 901.115659] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 901.123217] alg: No test for adler32 (adler32-zlib) [ 901.953388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 901.990157] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 901.992421] LNet: *** cfs_fail_loc=e003, val=100*** [ 902.340728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 902.353370] Ping reply received with mapping: [ 902.355283] found metadata reply NI[192.168.206.46@tcp] [ 902.357369] 2e ce a8 c0 00 00 02 00 5a 5a 5a 5a 5a 5a 5a 5a [ 902.359140] 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a [ 902.361024] 5a 5a 5a 5a 5a 5a 5a 5a [ 902.426903] LNet: 98670:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 902.429987] LNetError: 98670:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 902.445844] LNet: Removed LNI 192.168.206.46@tcp [ 902.754669] Key type .llcrypt unregistered [ 902.756984] Key type ._llcrypt unregistered [ 908.144037] Key type ._llcrypt registered [ 908.145950] Key type .llcrypt registered [ 908.891632] Lustre: DEBUG MARKER: == sanity-lnet test 280: Don't panic when request_module fails ========================================================== 23:46:02 (1781235962) [ 909.226380] Key type .llcrypt unregistered [ 909.227882] Key type ._llcrypt unregistered [ 912.160542] Key type ._llcrypt registered [ 912.161643] Key type .llcrypt registered [ 912.248141] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 912.257344] alg: No test for adler32 (adler32-zlib) [ 913.203675] LNetError: 99717:0:(api-ni.c:2729:lnet_load_lnd()) Can't load LND tcp, module ksocklnd, rc=256 [ 913.463474] Key type .llcrypt unregistered [ 913.467157] Key type ._llcrypt unregistered [ 916.165154] Key type ._llcrypt registered [ 916.166722] Key type .llcrypt registered [ 916.852482] Lustre: DEBUG MARKER: == sanity-lnet test 290: Check that lnet acceptor is using explicit address binding ========================================================== 23:46:10 (1781235970) [ 917.128486] Key type .llcrypt unregistered [ 917.130496] Key type ._llcrypt unregistered [ 920.310978] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 920.313640] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 922.600594] Key type ._llcrypt registered [ 922.602196] Key type .llcrypt registered [ 922.687819] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 922.701215] alg: No test for adler32 (adler32-zlib) [ 923.547406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 923.590532] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 928.424707] Lustre: DEBUG MARKER: == sanity-lnet test 291: load lnet using custom accept port for bulk tcp connections ========================================================== 23:46:22 (1781235982) [ 928.516915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 928.543362] LNet: 101496:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 928.550846] LNetError: 101496:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 928.569603] LNet: Removed LNI 10.1.2.3@tcp [ 928.586528] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 928.923083] Key type .llcrypt unregistered [ 928.924623] Key type ._llcrypt unregistered [ 931.343695] Key type ._llcrypt registered [ 931.345287] Key type .llcrypt registered [ 931.410367] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 931.418095] alg: No test for adler32 (adler32-zlib) [ 932.243745] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 932.279173] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 932.341388] LNet: 101784:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 932.344493] LNetError: 101784:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 932.354532] LNet: Removed LNI 192.168.206.46@tcp [ 932.612657] Key type .llcrypt unregistered [ 932.617418] Key type ._llcrypt unregistered [ 936.863563] Key type ._llcrypt registered [ 936.864645] Key type .llcrypt registered [ 937.405244] Lustre: DEBUG MARKER: == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 23:46:31 (1781235991) [ 937.644523] Key type .llcrypt unregistered [ 937.646168] Key type ._llcrypt unregistered [ 940.344562] Key type ._llcrypt registered [ 940.346017] Key type .llcrypt registered [ 940.422040] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 940.429274] alg: No test for adler32 (adler32-zlib) [ 943.898423] Lustre: DEBUG MARKER: == sanity-lnet test 301: Check for dynamic adds of same/wrong interface (memory leak) ========================================================== 23:46:37 (1781235997) [ 943.980221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 944.011658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 944.043518] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 944.054602] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 944.416597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 944.434918] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if ens2 [ 944.459773] LNet: Removed LNI 192.168.206.46@tcp [ 944.595130] LNet: 103526:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 944.767396] Key type .llcrypt unregistered [ 944.769036] Key type ._llcrypt unregistered [ 949.043198] Key type ._llcrypt registered [ 949.044191] Key type .llcrypt registered [ 949.664612] Lustre: DEBUG MARKER: == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 23:46:43 (1781236003) [ 949.831672] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 949.840234] alg: No test for adler32 (adler32-zlib) [ 950.656165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 950.682569] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 950.697501] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 951.019976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 951.036851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug peer --nid 192.168.206.46@tcp [ 951.048623] LNet: Conn 0 [type, closing, crefcnt, srefcnt]: 2, 0, 1, 1 [ 951.051229] LNet: Conn 0 rx [scheduled, ready, state]: 0, 0, 1 [ 951.052858] LNet: Conn 0 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 951, 0, 0 [ 951.055170] LNet: Conn 0 sched [nconns, cpt]: 4, 0 [ 951.056539] LNet: Conn 1 [type, closing, crefcnt, srefcnt]: 3, 0, 1, 1 [ 951.058437] LNet: Conn 1 rx [scheduled, ready, state]: 0, 0, 1 [ 951.060348] LNet: Conn 1 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 951, 0, 0 [ 951.063015] LNet: Conn 1 sched [nconns, cpt]: 4, 0 [ 951.064394] LNet: Conn 2 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 951.067011] LNet: Conn 2 rx [scheduled, ready, state]: 0, 0, 1 [ 951.069799] LNet: Conn 2 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 951, 0, 1001 [ 951.073753] LNet: Conn 2 sched [nconns, cpt]: 4, 0 [ 951.075229] LNet: Conn 3 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 951.077131] LNet: Conn 3 rx [scheduled, ready, state]: 0, 0, 1 [ 951.078666] LNet: Conn 3 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 951, 0, 1001 [ 951.081631] LNet: Conn 3 sched [nconns, cpt]: 4, 0 [ 953.590598] Lustre: DEBUG MARKER: == sanity-lnet test 303: Check peer NI health after link down ========================================================== 23:46:47 (1781236007) [ 953.781560] LNet: 105063:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 953.785848] LNetError: 105063:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 954.854476] LNet: Removed LNI 192.168.206.46@tcp [ 955.116668] Key type .llcrypt unregistered [ 955.118451] Key type ._llcrypt unregistered [ 957.362233] Key type ._llcrypt registered [ 957.364726] Key type .llcrypt registered [ 957.458854] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 957.468392] alg: No test for adler32 (adler32-zlib) [ 958.280394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 958.303371] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 961.836778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 965.266348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 965.282735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 965.293183] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 965.673397] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 965.706601] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 965.708971] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 965.711550] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 967.881434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp99 --if test1pg [ 967.891668] LNet: Added LNI 10.1.2.3@tcp99 [8/256/0/180] [ 972.188730] Lustre: DEBUG MARKER: == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 23:47:05 (1781236025) [ 972.262991] LNet: 106566:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 972.266870] LNetError: 106566:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 972.270848] LNet: Removed LNI 192.168.206.46@tcp [ 973.280150] LNet: Removed LNI 192.168.206.46@tcp1 [ 973.282151] LNetError: 106566:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 973.284700] LNetError: 106566:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 973.293749] LNet: Removed LNI 10.1.2.3@tcp99 [ 973.552300] Key type .llcrypt unregistered [ 973.553905] Key type ._llcrypt unregistered [ 976.020440] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 976.024689] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 978.234809] Key type ._llcrypt registered [ 978.236387] Key type .llcrypt registered [ 978.316038] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 978.324102] alg: No test for adler32 (adler32-zlib) [ 979.132416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 979.160910] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 979.181282] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 979.536309] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 979.545611] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 980.441064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.1.2.3@tcp --lock_prim [ 980.462140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 980.480644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 10.1.2.3@tcp --state 1048577 [ 980.492746] LNetError: 107311:0:(peer.c:2175:lnet_del_peer_ni()) peer 10.1.2.3@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 980.510385] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 980.529610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 980.708348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 980.736786] LNet: 107331:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 980.740714] LNetError: 107331:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 980.743993] LNet: Removed LNI 192.168.206.46@tcp [ 980.750336] LNet: Removed LNI 10.1.2.3@tcp [ 981.020605] Key type .llcrypt unregistered [ 981.022776] Key type ._llcrypt unregistered [ 985.164410] Key type ._llcrypt registered [ 985.165910] Key type .llcrypt registered [ 985.728585] Lustre: DEBUG MARKER: == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 23:47:19 (1781236039) [ 985.878983] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 985.885478] alg: No test for adler32 (adler32-zlib) [ 986.691210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 986.717723] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 986.729302] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 987.042676] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.46@tcp [ 987.079667] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping oleg646-client@tcp [ 989.562826] Lustre: DEBUG MARKER: == sanity-lnet test 310: Set timeout and verify ========== 23:47:23 (1781236043) [ 989.605485] LNet: 108857:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 989.609680] LNetError: 108857:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 989.626217] LNet: Removed LNI 192.168.206.46@tcp [ 989.894703] Key type .llcrypt unregistered [ 989.896436] Key type ._llcrypt unregistered [ 992.020711] Key type ._llcrypt registered [ 992.022539] Key type .llcrypt registered [ 992.093761] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 992.103077] alg: No test for adler32 (adler32-zlib) [ 992.917096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure --all [ 992.929985] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 992.943352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 995.531910] Lustre: DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 23:47:29 (1781236049) [ 996.141185] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type [ 996.784773] Lustre: DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 23:47:30 (1781236050) [ 997.324187] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type [ 997.943857] Lustre: DEBUG MARKER: == sanity-lnet test 350: Check import --exec of ping/discover ========================================================== 23:47:31 (1781236051) [ 997.992747] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 998.002812] LNet: 110280:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 998.007393] LNetError: 110280:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 998.018718] LNet: Removed LNI 192.168.206.46@tcp [ 998.027861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 998.054738] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 998.068166] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 998.374276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 998.395535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 1000.794824] Lustre: DEBUG MARKER: == sanity-lnet test 400: Check for udsp add/delete net rule without net num ========================================================== 23:47:34 (1781236054) [ 1000.858582] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1000.868473] LNet: 110991:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1000.871345] LNetError: 110991:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1000.878771] LNet: Removed LNI 192.168.206.46@tcp [ 1000.887826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1000.911612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --priority 0 [ 1000.926650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 1001.152208] Key type .llcrypt unregistered [ 1001.153270] Key type ._llcrypt unregistered [ 1005.132254] Key type ._llcrypt registered [ 1005.133784] Key type .llcrypt registered [ 1005.649797] Lustre: DEBUG MARKER: == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 23:47:39 (1781236059) [ 1005.813188] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1005.820258] alg: No test for adler32 (adler32-zlib) [ 1006.637560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1006.663466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1006.677319] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1006.993040] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 1007.018706] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1009.638587] Lustre: DEBUG MARKER: == sanity-lnet test 402: Destination net rule should not panic ========================================================== 23:47:43 (1781236063) [ 1009.689560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1009.700516] LNet: 112542:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1009.704675] LNetError: 112542:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1009.719619] LNet: Removed LNI 192.168.206.46@tcp [ 1009.732835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1009.755363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst kfi --priority 0 [ 1009.773448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 402@kfi [ 1012.458769] Lustre: DEBUG MARKER: == sanity-lnet test 403: Add and delete udsp net and nid rules ========================================================== 23:47:46 (1781236066) [ 1012.556281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1012.570517] LNet: 113125:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1012.580896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1012.610464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 1012.776253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 1012.794336] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 1012.822298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src 1.1.1.1@tcp --prio 1 [ 1012.990647] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 1013.009616] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 1013.055434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1013.076337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1013.110412] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 1013.267573] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 1013.281722] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 1013.309077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst 1.1.1.1@tcp --prio 1 [ 1013.475961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 1013.495310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 1016.508756] Lustre: DEBUG MARKER: == sanity-lnet test 404: Check udsp src/dst net rule ===== 23:47:50 (1781236070) [ 1016.572263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1016.586807] LNet: 113755:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1016.589934] LNet: 113755:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 1016.613488] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1016.639684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1016.657266] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1017.049664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1017.063336] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1017.602438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 1017.642049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1017.801154] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1017.817913] LNetError: 114036:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1017.823172] LNet: Removed LNI 192.168.206.46@tcp [ 1017.839093] LNet: Removed LNI 192.168.206.46@tcp2 [ 1017.854104] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1017.903610] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1017.911921] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1017.918682] LNet: 114042:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1018.146206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1018.248705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1019.296136] LNet: Removed LNI 192.168.206.46@tcp [ 1019.298107] LNetError: 114079:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1019.301660] LNetError: 114079:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 1019.312611] LNet: Removed LNI 192.168.206.46@tcp2 [ 1019.339828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1019.368481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1019.383883] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1019.771861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1019.786372] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1020.240344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1020.265481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 1020.296440] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1020.398994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1020.415627] LNet: 114364:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1020.420096] LNet: 114364:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 1021.472162] LNet: Removed LNI 192.168.206.46@tcp [ 1021.477244] LNetError: 114364:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1021.486659] LNetError: 114364:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 1021.497823] LNet: Removed LNI 192.168.206.46@tcp2 [ 1021.523982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1021.572336] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1021.577207] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1021.582209] LNet: 114370:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1021.821740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1021.948997] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1021.965211] LNet: Removed LNI 192.168.206.46@tcp [ 1023.016084] LNet: Removed LNI 192.168.206.46@tcp2 [ 1023.026503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1023.053950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1023.066119] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1023.419532] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1023.431991] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1023.829721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 1023.865825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1023.961316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1024.992107] LNet: Removed LNI 192.168.206.46@tcp [ 1025.001321] LNet: Removed LNI 192.168.206.46@tcp2 [ 1025.013533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1025.046405] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1025.049970] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1025.054078] LNet: 114693:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1025.294127] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1025.376291] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1025.386900] LNet: 114730:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1025.389683] LNet: 114730:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 1026.400127] LNet: Removed LNI 192.168.206.46@tcp [ 1026.401807] LNetError: 114730:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1026.405013] LNetError: 114730:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 5 previous similar messages [ 1026.414328] LNet: Removed LNI 192.168.206.46@tcp2 [ 1026.422516] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1026.446860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1026.462640] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1026.778016] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1026.790016] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1027.180611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1027.201989] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 1027.227392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1027.321487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1028.384252] LNet: Removed LNI 192.168.206.46@tcp [ 1029.416309] LNet: Removed LNI 192.168.206.46@tcp2 [ 1029.430178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1029.463513] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1029.468936] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1029.473128] LNet: 115020:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1029.689852] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1029.786387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1029.796343] LNet: Removed LNI 192.168.206.46@tcp [ 1030.823106] LNet: Removed LNI 192.168.206.46@tcp2 [ 1030.839596] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1030.881610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1030.900790] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1031.294277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1031.310345] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1031.809246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 1031.830495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 1031.863346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1032.029950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1033.056182] LNet: Removed LNI 192.168.206.46@tcp [ 1033.068351] LNet: Removed LNI 192.168.206.46@tcp2 [ 1033.084717] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1033.126509] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1033.130540] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1033.363589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1033.519763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1033.538115] LNet: 115412:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1033.543294] LNet: 115412:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 3 previous similar messages [ 1034.592138] LNet: Removed LNI 192.168.206.46@tcp [ 1034.595552] LNetError: 115412:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1034.598981] LNetError: 115412:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 7 previous similar messages [ 1034.608353] LNet: Removed LNI 192.168.206.46@tcp2 [ 1034.622468] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1034.648874] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1034.660436] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1034.988726] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 1034.999743] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1035.439456] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1035.459345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 1035.476632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 1035.505144] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1035.656902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1036.704146] LNet: Removed LNI 192.168.206.46@tcp [ 1037.734723] LNet: Removed LNI 192.168.206.46@tcp2 [ 1037.747938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1037.778605] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1037.787304] LNet: Added LNI 192.168.206.46@tcp2 [8/256/0/180] [ 1037.791127] LNet: 115719:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1037.795275] LNet: 115719:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 1038.036913] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1041.369882] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_410 skipping ALWAYS excluded test 410 [ 1042.171394] Lustre: DEBUG MARKER: == sanity-lnet test 450: Check import of multiple interfaces ========================================================== 23:48:15 (1781236095) [ 1042.219505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1042.231759] LNet: Removed LNI 192.168.206.46@tcp [ 1043.302500] LNet: Removed LNI 192.168.206.46@tcp2 [ 1043.317039] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1043.474996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1043.487220] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1043.845684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1043.855619] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1044.307878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1044.318251] LNet: Removed LNI 192.168.206.46@tcp [ 1044.324531] LNet: Removed LNI 10.1.2.3@tcp [ 1044.340373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1044.379797] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-450.yaml [ 1044.405367] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1044.411805] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1044.616429] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del /tmp/sanity-lnet-450.yaml [ 1044.630760] LNet: Removed LNI 192.168.206.46@tcp [ 1044.639923] LNet: Removed LNI 10.1.2.3@tcp [ 1047.621920] Lustre: DEBUG MARKER: == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 23:48:21 (1781236101) [ 1047.703368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1047.729547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1047.773561] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1047.839376] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1047.842273] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1047.844882] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1050.010993] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1050.027446] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1050.381096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1050.391400] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1051.424624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.46@tcp [ 1051.449447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d *@tcp -r 1 -l 1 -m PUT [ 1051.483344] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.206.46@tcp -m PUT -e local_timeout -r 1 [ 1051.518818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d 10.1.2.3@tcp -m PUT -e local_timeout -r 1 [ 1051.562624] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1051.564993] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1051.690242] LNetError: 117413:0:(lib-move.c:4710:lnet_parse()) 192.168.206.46@tcp, src 192.168.206.46@tcp: Dropping PUT (error -108 looking up sender) [ 1052.707405] LNet: 117873:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1052.712758] LNet: 117873:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 4 previous similar messages [ 1052.716435] LNetError: 117873:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1052.719603] LNetError: 117873:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 6 previous similar messages [ 1052.723788] LNet: Removed LNI 192.168.206.46@tcp [ 1052.736276] LNet: Removed LNI 10.1.2.3@tcp [ 1052.938071] Key type .llcrypt unregistered [ 1052.939633] Key type ._llcrypt unregistered [ 1057.459137] Key type ._llcrypt registered [ 1057.460897] Key type .llcrypt registered [ 1058.012421] Lustre: DEBUG MARKER: == sanity-lnet test 501: Verify lnetctl peer set --health (SR) ========================================================== 23:48:31 (1781236111) [ 1058.152748] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1058.159199] alg: No test for adler32 (adler32-zlib) [ 1058.980985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1059.137769] LNet: 118725:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1059.384424] Key type .llcrypt unregistered [ 1059.385732] Key type ._llcrypt unregistered [ 1061.750552] Key type ._llcrypt registered [ 1061.752116] Key type .llcrypt registered [ 1061.845349] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1061.854814] alg: No test for adler32 (adler32-zlib) [ 1062.675529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1062.689747] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1066.127312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 1066.200124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1066.450365] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.206.146@tcp [ 1066.547546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1066.563296] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1066.581392] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1066.600289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1066.615786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1066.630853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1066.649654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1066.664697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1066.674562] LNet: 119173:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1066.687955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1066.704487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1066.723060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1066.741885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1066.760987] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1066.777659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1067.042483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 1067.149521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1067.164129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1067.181705] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1067.197857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1067.213635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1067.228655] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1067.249197] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1067.269267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1067.286625] LNet: 119290:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1067.298817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1067.317588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1067.338227] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1067.358526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1067.379215] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1067.404787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1067.684548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.206.146@tcp [ 1067.793453] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1067.809110] LNet: 1 peer NIs in recovery (showing 1): 192.168.206.146@tcp [ 1067.812636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1067.832038] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1067.848289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1067.863986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1067.882239] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1067.899096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1067.914808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1067.932102] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1067.950520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1067.965746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1067.984962] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1068.007462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1068.025581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1068.296045] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 1068.404264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1068.419539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1068.439162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1068.464473] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1068.483269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1068.499651] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1068.518490] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1068.537564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1068.546493] LNet: 119524:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1068.551630] LNet: 119524:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 1068.560480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1068.577811] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1068.593890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1068.613848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1068.634626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1068.651508] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1068.796904] LNet: 119570:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1068.800757] LNetError: 119570:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1068.810705] LNet: Removed LNI 192.168.206.46@tcp [ 1068.996724] Key type .llcrypt unregistered [ 1068.998204] Key type ._llcrypt unregistered [ 1075.513794] Key type ._llcrypt registered [ 1075.515236] Key type .llcrypt registered [ 1076.220876] Lustre: DEBUG MARKER: == sanity-lnet test 502: Verify lnetctl peer set --health (MR) ========================================================== 23:48:49 (1781236129) [ 1076.372249] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1076.380828] alg: No test for adler32 (adler32-zlib) [ 1077.196354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1077.372412] LNet: 120455:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1077.687300] Key type .llcrypt unregistered [ 1077.688579] Key type ._llcrypt unregistered [ 1080.073122] Key type ._llcrypt registered [ 1080.074501] Key type .llcrypt registered [ 1080.163978] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1080.172867] alg: No test for adler32 (adler32-zlib) [ 1081.006617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1081.029486] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1084.436308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.146@tcp [ 1087.875465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1087.894737] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1087.905734] LNet: Added LNI 192.168.206.46@tcp1 [8/256/0/180] [ 1088.332101] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1088.555278] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.206.146@tcp [ 1088.648232] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1088.664099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1088.680352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1088.696650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1088.714798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1088.730177] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1088.746194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1088.763245] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1088.772688] LNet: 121105:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1088.783250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1088.800513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1088.817983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1088.835121] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1088.852228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1088.867380] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1089.126851] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 1089.234319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1089.248229] LNet: 2 peer NIs in recovery (showing 2): 192.168.206.146@tcp, 192.168.206.146@tcp1 [ 1089.251713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1089.270664] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1089.288745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1089.305757] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1089.324621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1089.341764] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1089.357889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1089.368294] LNet: 121222:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1089.378077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1089.399911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1089.418727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1089.434607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1089.450725] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1089.467927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1089.724570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.206.146@tcp [ 1089.825666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1089.842755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1089.860166] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1089.875765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1089.896157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1089.912341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1089.929448] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1089.946262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1089.961916] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1089.976636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1089.992853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1090.009762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1090.026063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1090.043610] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1090.296673] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 1090.389861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1090.404396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1090.420214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1090.439817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1090.454997] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1090.471170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1090.493733] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1090.511088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1090.519752] LNet: 121456:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1090.523662] LNet: 121456:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 1090.534263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1090.551135] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1090.567746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1090.586861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1090.603646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1090.622244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1093.128849] LNet: 121537:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1093.134269] LNetError: 121537:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1093.136860] LNet: Removed LNI 192.168.206.46@tcp [ 1093.146217] LNet: Removed LNI 192.168.206.46@tcp1 [ 1093.336549] Key type .llcrypt unregistered [ 1093.338166] Key type ._llcrypt unregistered [ 1100.187622] Key type ._llcrypt registered [ 1100.189335] Key type .llcrypt registered [ 1100.771748] Lustre: DEBUG MARKER: == sanity-lnet test 600: lnet_legacy2yaml: basic ip2nets conversion ========================================================== 23:49:14 (1781236154) [ 1103.808265] Lustre: DEBUG MARKER: == sanity-lnet test 601: lnet_legacy2yaml: basic routes conversion ========================================================== 23:49:17 (1781236157) [ 1107.011972] Lustre: DEBUG MARKER: == sanity-lnet test 602: lnet_legacy2yaml: routes with priority ========================================================== 23:49:20 (1781236160) [ 1109.852533] Lustre: DEBUG MARKER: == sanity-lnet test 603: lnet_legacy2yaml: default hop flag ========================================================== 23:49:23 (1781236163) [ 1112.823232] Lustre: DEBUG MARKER: == sanity-lnet test 604: lnet_legacy2yaml: default hop with explicit value ========================================================== 23:49:26 (1781236166) [ 1115.839318] Lustre: DEBUG MARKER: == sanity-lnet test 605: lnet_legacy2yaml: quote leading star in IP pattern ========================================================== 23:49:29 (1781236169) [ 1118.884450] Lustre: DEBUG MARKER: == sanity-lnet test 606: lnet_legacy2yaml: file inputs === 23:49:32 (1781236172) [ 1121.661457] Lustre: DEBUG MARKER: == sanity-lnet test 607: lnet_legacy2yaml: options line with combined ip2nets and routes ========================================================== 23:49:35 (1781236175) [ 1124.416479] Lustre: DEBUG MARKER: == sanity-lnet test 608: lnet_legacy2yaml: error on invalid rule (missing IP pattern) ========================================================== 23:49:38 (1781236178) [ 1127.104053] Lustre: DEBUG MARKER: == sanity-lnet test 609: lnet_legacy2yaml: error on no input ========================================================== 23:49:40 (1781236180) [ 1129.759864] Lustre: DEBUG MARKER: == sanity-lnet test 610: lnet_legacy2yaml: multiple files and inline, check order ========================================================== 23:49:43 (1781236183) [ 1132.628074] Lustre: DEBUG MARKER: == sanity-lnet test 611: lnet_legacy2yaml: invalid hopcount ========================================================== 23:49:46 (1781236186) [ 1135.404982] Lustre: DEBUG MARKER: == sanity-lnet test 612: lnet_legacy2yaml: malformed IP pattern ========================================================== 23:49:49 (1781236189) [ 1138.407241] Lustre: DEBUG MARKER: == sanity-lnet test 613: lnet_legacy2yaml: multiple IP patterns in single rule ========================================================== 23:49:52 (1781236192) [ 1141.260156] Lustre: DEBUG MARKER: == sanity-lnet test 614: lnet_legacy2yaml: interfaces enumeration with multiple IP ranges ========================================================== 23:49:55 (1781236195) [ 1144.219889] Lustre: DEBUG MARKER: == sanity-lnet test 615: lnet_legacy2yaml: modprobe configuration file input ========================================================== 23:49:57 (1781236197) [ 1147.124853] Lustre: DEBUG MARKER: == sanity-lnet test 616: lnet_legacy2yaml: combining modprobe file with explicit ip2nets file ========================================================== 23:50:00 (1781236200) [ 1150.000666] Lustre: DEBUG MARKER: == sanity-lnet test 617: lnet_legacy2yaml: round-trip tunables from LNet to modprobe to YAML ========================================================== 23:50:03 (1781236203) [ 1150.173142] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1150.180053] alg: No test for adler32 (adler32-zlib) [ 1151.001028] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1151.065519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1151.084326] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1151.606074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1151.618936] LNet: 132232:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1151.624251] LNetError: 132232:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1151.635417] LNet: Removed LNI 192.168.206.46@tcp [ 1151.645908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1151.672743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 1151.684347] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1154.681933] Lustre: DEBUG MARKER: == sanity-lnet test 618: lnet_legacy2yaml: alternative route format ========================================================== 23:50:08 (1781236208) [ 1157.831954] Lustre: DEBUG MARKER: == sanity-lnet test 619: lnet_legacy2yaml: multiple alternative format routes from file ========================================================== 23:50:11 (1781236211) [ 1161.011344] Lustre: DEBUG MARKER: == sanity-lnet test 620: lnet_legacy2yaml: mixed legacy and alternative route formats ========================================================== 23:50:14 (1781236214) [ 1164.025986] Lustre: DEBUG MARKER: == sanity-lnet test 621: lnet_legacy2yaml: alternative format with invalid hopcount ========================================================== 23:50:17 (1781236217) [ 1166.715967] Lustre: DEBUG MARKER: == sanity-lnet test 622: lnet_legacy2yaml: alternative format with default-hop flag ========================================================== 23:50:20 (1781236220) [ 1169.492189] Lustre: DEBUG MARKER: == sanity-lnet test 623: lnet_legacy2yaml: basic networks conversion ========================================================== 23:50:23 (1781236223) [ 1172.679306] Lustre: DEBUG MARKER: == sanity-lnet test 624: lnet_legacy2yaml: networks with interfaces ========================================================== 23:50:26 (1781236226) [ 1175.956773] Lustre: DEBUG MARKER: == sanity-lnet test 625: lnet_legacy2yaml: multiple networks ========================================================== 23:50:29 (1781236229) [ 1178.987496] Lustre: DEBUG MARKER: == sanity-lnet test 626: lnet_legacy2yaml: mixed networks, ip2nets, and routes ========================================================== 23:50:32 (1781236232) [ 1182.064100] Lustre: DEBUG MARKER: == sanity-lnet test 627: lnet_legacy2yaml: invalid networks format ========================================================== 23:50:35 (1781236235) [ 1184.974215] Lustre: DEBUG MARKER: == sanity-lnet test 628: lnet_legacy2yaml: missing delimiter error ========================================================== 23:50:38 (1781236238) [ 1187.797564] Lustre: DEBUG MARKER: == sanity-lnet test 629: lnet_legacy2yaml: multiple gateways error ========================================================== 23:50:41 (1781236241) [ 1190.685527] Lustre: DEBUG MARKER: == sanity-lnet test complete, duration 1059 sec ========== 23:50:44 (1781236244) [ 1190.809131] LNet: 139687:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1190.813187] LNetError: 139687:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1190.826509] LNet: Removed LNI 192.168.206.46@tcp [ 1191.114650] Key type .llcrypt unregistered [ 1191.116559] Key type ._llcrypt unregistered [ 1195.920337] Key type ._llcrypt registered [ 1195.923094] Key type .llcrypt registered [ 1196.086825] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1196.094820] alg: No test for adler32 (adler32-zlib) [ 1196.978253] Lustre: Lustre: Build Version: 2.17.53_82_geaee21b [ 1197.100638] LNet: Added LNI 192.168.206.46@tcp [8/256/0/180] [ 1198.704360] Key type lgssc registered [ 1199.307846] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1225.346669] Lustre: Mounted lustre-client [ 1228.121209] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1231.198313] Lustre: Unmounted lustre-client [ 1279.487637] Key type lgssc unregistered [ 1279.617411] LNet: 142608:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1279.622468] LNetError: 142608:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1279.632714] LNet: Removed LNI 192.168.206.46@tcp [ 1279.928601] Key type .llcrypt unregistered [ 1279.930698] Key type ._llcrypt unregistered