[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd9fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffda000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 403941018 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffda max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54d0-0x000f54df] [ 0.000000] RAMDISK: [mem 0xbcc64000-0xbffcffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52F0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2464 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2300 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022C0 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2374 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE2404 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE243C 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2300-0xbffe2373] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22ff] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2374-0xbffe2403] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe2404-0xbffe243b] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe243c-0xbffe2463] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffd9fff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4744 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffda000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059618 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829700K/4306400K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003100] x2apic enabled [ 0.004007] Switched APIC routing to physical x2apic. [ 0.005013] kvm-guest: setup PV IPIs [ 0.008314] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009018] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010010] pid_max: default: 32768 minimum: 301 [ 0.011121] LSM: Security Framework initializing [ 0.012047] Yama: becoming mindful. [ 0.013073] SELinux: Initializing. [ 0.015002] *** VALIDATE selinux *** [ 0.023071] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028141] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029173] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031103] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032135] *** VALIDATE tmpfs *** [ 0.033520] *** VALIDATE proc *** [ 0.035216] *** VALIDATE cgroup *** [ 0.036007] *** VALIDATE cgroup2 *** [ 0.037305] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039061] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040006] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041035] Spectre V2 : User space: Vulnerable [ 0.042008] Speculative Store Bypass: Vulnerable [ 0.045489] debug: unmapping init [mem 0xffffffffbcc59000-0xffffffffbcc60fff] [ 0.047826] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048714] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049018] ... version: 2 [ 0.050012] ... bit width: 48 [ 0.051009] ... generic registers: 4 [ 0.051964] ... value mask: 0000ffffffffffff [ 0.052015] ... max period: 00007fffffffffff [ 0.053016] ... fixed-purpose events: 3 [ 0.054010] ... event mask: 000000070000000f [ 0.056228] rcu: Hierarchical SRCU implementation. [ 0.058467] smp: Bringing up secondary CPUs ... [ 0.059593] x86: Booting SMP configuration: [ 0.060020] .... node #0, CPUs: #1 #2 #3 [ 0.063396] smp: Brought up 1 node, 4 CPUs [ 0.065014] smpboot: Max logical packages: 1 [ 0.066018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.260022] node 0 deferred pages initialised in 192ms [ 0.263144] devtmpfs: initialized [ 0.264261] x86/mm: Memory block size: 128MB [ 0.266274] gcov: version magic: 0x41383552 [ 0.270043] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.271084] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.273280] pinctrl core: initialized pinctrl subsystem [ 0.275108] [ 0.275413] ************************************************************* [ 0.277009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.279012] ** ** [ 0.281013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.283010] ** ** [ 0.285012] ** This means that this kernel is built to expose internal ** [ 0.287009] ** IOMMU data structures, which may compromise security on ** [ 0.289009] ** your system. ** [ 0.290007] ** ** [ 0.292008] ** If you see this message and you are not debugging the ** [ 0.294011] ** kernel, report this immediately to your vendor! ** [ 0.296012] ** ** [ 0.299013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.301021] ************************************************************* [ 0.303568] NET: Registered protocol family 16 [ 0.305407] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.308128] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.311092] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.314150] cpuidle: using governor menu [ 0.316314] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.319472] PCI: Using configuration type 1 for base access [ 0.322131] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.330110] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.332045] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.336077] cryptd: max_cpu_qlen set to 1000 [ 0.337223] ACPI: Added _OSI(Module Device) [ 0.338030] ACPI: Added _OSI(Processor Device) [ 0.339014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.341012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.345174] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.351389] ACPI: Interpreter enabled [ 0.353072] ACPI: PM: (supports S0 S3 S4 S5) [ 0.355013] ACPI: Using IOAPIC for interrupt routing [ 0.356143] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.360694] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.371172] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.373038] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.376022] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.379067] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.384449] acpiphp: Slot [2] registered [ 0.386109] acpiphp: Slot [5] registered [ 0.387079] acpiphp: Slot [6] registered [ 0.389162] acpiphp: Slot [3] registered [ 0.390123] acpiphp: Slot [4] registered [ 0.392101] acpiphp: Slot [7] registered [ 0.393078] acpiphp: Slot [8] registered [ 0.395084] acpiphp: Slot [9] registered [ 0.396069] acpiphp: Slot [10] registered [ 0.397080] acpiphp: Slot [11] registered [ 0.399093] acpiphp: Slot [12] registered [ 0.400069] acpiphp: Slot [13] registered [ 0.401090] acpiphp: Slot [14] registered [ 0.402100] acpiphp: Slot [15] registered [ 0.404062] acpiphp: Slot [16] registered [ 0.405057] acpiphp: Slot [17] registered [ 0.406104] acpiphp: Slot [18] registered [ 0.408097] acpiphp: Slot [19] registered [ 0.409103] acpiphp: Slot [20] registered [ 0.411111] acpiphp: Slot [21] registered [ 0.413109] acpiphp: Slot [22] registered [ 0.414081] acpiphp: Slot [23] registered [ 0.416089] acpiphp: Slot [24] registered [ 0.417084] acpiphp: Slot [25] registered [ 0.418138] acpiphp: Slot [26] registered [ 0.419102] acpiphp: Slot [27] registered [ 0.421086] acpiphp: Slot [28] registered [ 0.422081] acpiphp: Slot [29] registered [ 0.424102] acpiphp: Slot [30] registered [ 0.425080] acpiphp: Slot [31] registered [ 0.426052] PCI host bridge to bus 0000:00 [ 0.427018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.429020] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.432021] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.435023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.437015] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.439020] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.441070] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.443776] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.447073] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.455014] pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] [ 0.458087] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.461018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.464016] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.466013] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.469648] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.472789] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.475030] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.478684] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.483013] pci 0000:00:02.0: reg 0x10: [io 0xc100-0xc11f] [ 0.491837] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.496012] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.501086] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.505736] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.510014] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.523019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.533081] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.539016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.546016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.561033] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.569785] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.570455] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.573403] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.576412] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.578222] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.583170] iommu: Default domain type: Passthrough [ 0.584599] SCSI subsystem initialized [ 0.586143] ACPI: bus type USB registered [ 0.587107] usbcore: registered new interface driver usbfs [ 0.589092] usbcore: registered new interface driver hub [ 0.591133] usbcore: registered new device driver usb [ 0.593160] pps_core: LinuxPPS API ver. 1 registered [ 0.595013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.598097] PTP clock support registered [ 0.600187] EDAC MC: Ver: 3.0.0 [ 0.602171] PCI: Using ACPI for IRQ routing [ 0.604707] NetLabel: Initializing [ 0.606010] NetLabel: domain hash size = 128 [ 0.607010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.609099] NetLabel: unlabeled traffic allowed by default [ 0.611042] vgaarb: loaded [ 0.612244] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.614015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.619438] clocksource: Switched to clocksource kvm-clock [ 0.721855] VFS: Disk quotas dquot_6.6.0 [ 0.723825] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.726408] *** VALIDATE ramfs *** [ 0.727673] *** VALIDATE hugetlbfs *** [ 0.729185] pnp: PnP ACPI init [ 0.731401] pnp: PnP ACPI: found 6 devices [ 0.748417] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.750730] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.753138] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.755429] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.757566] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.760130] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.763435] NET: Registered protocol family 2 [ 0.766089] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.772647] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.776250] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.781694] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.784779] TCP: Hash tables configured (established 65536 bind 65536) [ 0.787392] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.790253] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.792954] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.796184] NET: Registered protocol family 1 [ 0.798581] RPC: Registered named UNIX socket transport module. [ 0.800900] RPC: Registered udp transport module. [ 0.802283] RPC: Registered tcp transport module. [ 0.803887] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.806105] NET: Registered protocol family 44 [ 0.807322] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.810362] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.812823] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.815186] PCI: CLS 0 bytes, default 64 [ 0.816491] Unpacking initramfs... [ 2.119806] debug: unmapping init [mem 0xffff97fbfcc64000-0xffff97fbfffcffff] [ 2.123164] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.125426] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.128134] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.601940] Initialise system trusted keyrings [ 2.603690] Key type blacklist registered [ 2.605411] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.614231] zbud: loaded [ 2.616981] *** VALIDATE nfs *** [ 2.618127] *** VALIDATE nfs4 *** [ 2.619556] pstore: using deflate compression [ 2.622425] Platform Keyring initialized [ 2.717095] NET: Registered protocol family 38 [ 2.718743] Key type asymmetric registered [ 2.720214] Asymmetric key parser 'x509' registered [ 2.721949] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.724714] io scheduler mq-deadline registered [ 2.726162] io scheduler kyber registered [ 2.727591] io scheduler bfq registered [ 2.729357] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.731563] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.734146] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.736904] ACPI: Power Button [PWRF] [ 2.741665] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.747322] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.757743] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.785670] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.812818] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.818120] Non-volatile memory driver v1.3 [ 2.819398] Linux agpgart interface v0.103 [ 2.848419] virtio_blk virtio1: [vda] 136464 512-byte logical blocks (69.9 MB/66.6 MiB) [ 2.851341] vda: detected capacity change from 0 to 69869568 [ 2.863752] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.866235] vdb: detected capacity change from 0 to 1073741824 [ 2.873158] libphy: Fixed MDIO Bus: probed [ 2.878675] usbcore: registered new interface driver usbserial_generic [ 2.881025] usbserial: USB Serial support registered for generic [ 2.883249] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.887223] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.888844] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.890886] mousedev: PS/2 mouse device common for all mice [ 2.893818] rtc_cmos 00:05: RTC can wake from S4 [ 2.896692] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.897765] rtc_cmos 00:05: registered as rtc0 [ 2.901081] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.903982] intel_pstate: CPU model not supported [ 2.906553] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.908521] hid: raw HID events driver (C) Jiri Kosina [ 2.910951] usbcore: registered new interface driver usbhid [ 2.912577] usbhid: USB HID core driver [ 2.912798] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.913717] drop_monitor: Initializing network drop monitor service [ 2.917610] Initializing XFRM netlink socket [ 2.919402] NET: Registered protocol family 10 [ 2.922782] Segment Routing with IPv6 [ 2.924626] NET: Registered protocol family 17 [ 2.926875] mpls_gso: MPLS GSO support [ 2.932150] RAS: Correctable Errors collector initialized. [ 2.934111] AVX version of gcm_enc/dec engaged. [ 2.935377] AES CTR mode by8 optimization enabled [ 3.019242] sched_clock: Marking stable (3019190362, 0)->(3832472817, -813282455) [ 3.024360] registered taskstats version 1 [ 3.026196] Loading compiled-in X.509 certificates [ 3.029327] zswap: loaded using pool lzo/zbud [ 3.053473] Key type big_key registered [ 3.066120] Key type encrypted registered [ 3.067822] ima: No TPM chip found, activating TPM-bypass! [ 3.069685] ima: Allocated hash algorithm: sha1 [ 3.071247] ima: No architecture policies found [ 3.072688] evm: Initialising EVM extended attributes: [ 3.074272] evm: security.selinux [ 3.075296] evm: security.ima [ 3.076121] evm: security.capability [ 3.077290] evm: HMAC attrs: 0x1 [ 3.079242] rtc_cmos 00:05: setting system clock to 2026-06-12 01:52:57 UTC (1781229177) [ 3.084384] debug: unmapping init [mem 0xffffffffbdc03000-0xffffffffbddfffff] [ 3.087327] debug: unmapping init [mem 0xffffffffbc982000-0xffffffffbcc58fff] [ 3.100110] Write protecting the kernel read-only data: 28672k [ 3.103469] debug: unmapping init [mem 0xffffffffbb003000-0xffffffffbb1fffff] [ 3.106399] debug: unmapping init [mem 0xffffffffbb914000-0xffffffffbb9fffff] [ 3.138606] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.146334] systemd[1]: Detected virtualization kvm. [ 3.148345] systemd[1]: Detected architecture x86-64. [ 3.150423] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.180172] systemd[1]: No hostname configured. [ 3.182280] systemd[1]: Set hostname to . [ 3.184734] random: systemd: uninitialized urandom read (16 bytes read) [ 3.187649] systemd[1]: Initializing machine ID from random generator. [ 3.235877] random: ln: uninitialized urandom read (6 bytes read) [ 3.326118] random: systemd: uninitialized urandom read (16 bytes read) [ 3.329855] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.336376] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.342675] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Swap. [ OK ] Reached target Local File Systems. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. Starting Journal Service... Starting Create Volatile Files and Directories... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Timers. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Journal Service. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.956215] device-mapper: uevent: version 1.0.3 [ 3.958397] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.669579] virtio_net virtio0 ens2: renamed from eth0 [ 4.725785] scsi host0: ata_piix [ 4.739294] scsi host1: ata_piix [ 4.741433] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 [ 4.743758] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 [ 5.843111] random: fast init done [ 9.017140] dracut-initqueue[587]: RTNETLINK answers: File exists [ 9.602708] random: crng init done [ 9.603967] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 9.879175] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Timers. [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 10.948173] printk: systemd: 22 output lines suppressed due to ratelimiting [ 11.188670] SELinux: Disabled at runtime. [ 11.247106] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.257232] systemd[1]: Detected virtualization kvm. [ 11.259375] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.709414] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.712783] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.717722] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.723440] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.727020] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.735413] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.750908] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. Mounting Kernel Debug File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on RPCbind Server Activation Socket. Activating swap /dev/disk/by-label/SWAP... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-getty.slice. Mounting POSIX Message Queue File System... Starting Remount Root and Kernel File Systems... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 11.816241] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Apply Kernel Variables... [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Huge Pages File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 12.211385] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.470861] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.553537] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.636583] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.653955] EDAC sbridge: Ver: 1.1.2 [ 13.802621] Key type dns_resolver registered [ 14.121792] NFS: Registering the id_resolver key type [ 14.123672] Key type id_resolver registered [ 14.125424] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Authorization Manager. [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg620-client login: [ 43.182340] libcfs: loading out-of-tree module taints kernel. [ 43.205870] Key type ._llcrypt registered [ 43.207121] Key type .llcrypt registered [ 43.432415] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 43.440731] alg: No test for adler32 (adler32-zlib) [ 44.505224] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 44.925668] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 46.561728] Key type lgssc registered [ 47.441923] Lustre: Echo OBD driver; http://www.lustre.org/ [ 109.062634] Lustre: Mounted lustre-client [ 112.273979] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 119.982627] Lustre: DEBUG MARKER: oleg620-client.virtnet: executing check_logdir /tmp/testlogs/ [ 122.224812] Lustre: DEBUG MARKER: oleg620-client.virtnet: executing yml_node [ 124.358384] Lustre: DEBUG MARKER: Client: 2.17.53.61 [ 125.601668] Lustre: DEBUG MARKER: MDS: 2.17.53.61 [ 126.773822] Lustre: DEBUG MARKER: OSS: 2.17.53.61 [ 127.480406] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Thu Jun 11 21:55:01 EDT 2026 [ 134.105272] Lustre: Unmounted lustre-client [ 146.934870] Key type lgssc unregistered [ 147.081969] LNet: 8370:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 147.087322] LNetError: 8370:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 147.096581] LNet: Removed LNI 192.168.206.20@tcp [ 147.434285] Key type .llcrypt unregistered [ 147.436293] Key type ._llcrypt unregistered [ 158.305688] Key type ._llcrypt registered [ 158.307240] Key type .llcrypt registered [ 158.382172] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 158.391111] alg: No test for adler32 (adler32-zlib) [ 159.198971] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 159.227594] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 159.240862] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show [ 160.976318] Lustre: DEBUG MARKER: excepting tests: 253 254 410 [ 161.016489] LNet: 9244:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 161.019942] LNetError: 9244:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 161.031066] LNet: Removed LNI 192.168.206.20@tcp [ 161.353575] Key type .llcrypt unregistered [ 161.355263] Key type ._llcrypt unregistered [ 164.025420] Key type ._llcrypt registered [ 164.026574] Key type .llcrypt registered [ 164.653378] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 21:55:38 (1781229338) [ 164.819824] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 164.826749] alg: No test for adler32 (adler32-zlib) [ 165.642872] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 165.680996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 165.692722] LNet: 9661:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 165.702381] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 165.729066] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 165.739587] LNet: 9668:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 168.444540] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 21:55:42 (1781229342) [ 168.486581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 168.500987] LNet: 10241:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 168.517068] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 168.566390] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp [ 171.547942] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 21:55:45 (1781229345) [ 171.605480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 171.619916] LNet: 10827:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 171.634187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 171.672190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 2.2.2.2@o2ib [ 174.625630] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 21:55:48 (1781229348) [ 174.668465] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 174.679737] LNet: 11407:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 174.692961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 174.739556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib [ 177.440793] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 21:55:51 (1781229351) [ 177.475525] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 177.498765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 177.536628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp [ 177.715675] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp [ 180.471693] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 21:55:54 (1781229354) [ 180.544092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 180.561603] LNet: 12586:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 180.565506] LNet: 12586:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 180.578254] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 180.621354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 180.832397] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib [ 181.025545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib [ 183.840031] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 21:55:57 (1781229357) [ 183.897190] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 183.920259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 183.959184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi,[6-12/4]@bxi3f [ 186.751269] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 21:56:00 (1781229360) [ 186.801849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 186.830618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 186.873983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp [ 186.893460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 187.092265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib [ 187.115576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 187.299443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp [ 187.321287] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 187.500228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib [ 187.521709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@o2ib [ 187.693474] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib [ 187.710550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7.7.7.7@tcp [ 187.889938] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni [ 187.910915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 188.082280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni [ 188.100404] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 188.265546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi [ 188.282145] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 188.444607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi [ 188.460315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@kfi [ 188.616821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f [ 188.633801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 188.827693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@bxi3f --nid [8-12]@bxi3f [ 188.847303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@bxi3f [ 189.014912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi,[3-7]@bxi3f [ 189.033994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 7@gni [ 189.200980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 189.220165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp [ 191.977710] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 21:56:05 (1781229365) [ 192.025495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 192.036642] LNet: 14443:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 192.040242] LNet: 14443:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 2 previous similar messages [ 192.055128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 192.100912] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp [ 192.119882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp [ 194.843528] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 21:56:08 (1781229368) [ 194.884623] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 194.906184] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 194.947061] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 194.966347] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp [ 197.787166] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 21:56:11 (1781229371) [ 197.880261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 197.896654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 197.930066] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp [ 197.946318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp [ 200.882098] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 21:56:14 (1781229374) [ 200.954769] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 200.991387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 201.037288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 201.070976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp [ 203.915211] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 21:56:17 (1781229377) [ 203.976038] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 204.000653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 204.039685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp [ 204.064765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp [ 206.845612] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 21:56:20 (1781229380) [ 206.889242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 206.911169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 206.948904] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 206.966878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib [ 209.767576] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 21:56:23 (1781229383) [ 209.814731] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 209.827097] LNet: 17954:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 209.831476] LNet: 17954:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 209.848235] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 209.907262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 210.104863] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 213.020874] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 21:56:26 (1781229386) [ 213.066302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 213.090666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 213.187537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 213.294754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 215.984283] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 21:56:29 (1781229389) [ 216.035214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 216.056885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 216.112976] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 216.210795] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 218.772080] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 21:56:32 (1781229392) [ 218.834569] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 218.853836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 219.042205] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 219.149857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 221.604888] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 21:56:35 (1781229395) [ 221.640980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 221.658902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 221.694458] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 221.716398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 224.135474] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 21:56:38 (1781229398) [ 224.167524] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 224.186288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 224.223841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 224.247856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del [ 226.762074] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 21:56:40 (1781229400) [ 226.789857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 226.808738] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 226.847350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 19@gni [ 229.616822] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 21:56:43 (1781229403) [ 229.668346] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 229.700215] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 229.745588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib [ 232.786990] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 21:56:46 (1781229406) [ 232.838065] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 232.868893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 232.911466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni [ 233.096845] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni [ 236.146369] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 21:56:49 (1781229409) [ 236.184474] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 236.211295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 236.263269] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni [ 236.279274] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 22@gni --nid 26@gni [ 238.963546] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 21:56:52 (1781229412) [ 238.995567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 239.018125] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 239.062659] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni [ 239.079447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni [ 241.715810] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 21:56:55 (1781229415) [ 241.751768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 241.769398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 241.812948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni [ 241.831285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp [ 244.421962] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 21:56:58 (1781229418) [ 244.451424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 244.460616] LNet: 25145:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 244.463527] LNet: 25145:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 11 previous similar messages [ 244.477911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 244.513255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 244.533263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni [ 247.254642] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 21:57:01 (1781229421) [ 247.354670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 247.373898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 247.400877] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim [ 247.433405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp [ 247.442567] LNetError: 25748:0:(peer.c:2175:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 247.461325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@tcp --force [ 247.477728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show --nid 1.1.1.1@tcp [ 249.932039] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 21:57:03 (1781229423) [ 249.981389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 249.998389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 250.035773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.gPqbHUfkR2 [ 250.069858] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.gPqbHUfkR2 [ 250.094410] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/tmp.gPqbHUfkR2 [ 252.574380] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 21:57:06 (1781229426) [ 252.640165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 252.662382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 252.692660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp [ 252.716294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp [ 252.740719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer list [ 255.401491] Lustre: DEBUG MARKER: == sanity-lnet test 50: Enable/disable routing via yaml import ========================================================== 21:57:09 (1781229429) [ 255.447671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 255.467264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 255.502575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 256.563489] LNet: Message forwarding will be enabled in 2s [ 258.592164] LNet: Message forwarding enabled [ 258.722273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 258.735169] LNet: Message forwarding will stop in 2s to 6s [ 262.934886] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-50.yaml [ 263.421608] LNet: Message forwarding disabled [ 264.006227] LNet: Message forwarding will be enabled in 2s [ 266.325745] Lustre: DEBUG MARKER: == sanity-lnet test 51: Set numa range via yaml import === 21:57:20 (1781229440) [ 266.360607] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 266.941312] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 266.976195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-51.yaml [ 269.377356] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 21:57:23 (1781229443) [ 269.485218] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 269.504333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 269.531837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid foobar [ 269.549925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid foobar [ 269.569709] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --prim_nid 1.1.1.1@o2ib [ 269.587946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer del --nid 1.1.1.1@tcp [ 269.603970] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --nid 1.1.1.1@tcp [ 269.619519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add [ 269.635295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 269.651398] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp [ 269.672783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@foo [ 269.693742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@tcp [ 269.711946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1@o2ib [ 269.728371] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@gni [ 269.744396] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni [ 269.762038] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni [ 269.778978] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp [ 269.797770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp [ 269.817787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp [ 269.836373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar [ 269.853442] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp --gateway 1@gni [ 269.863795] LNetError: 28805:0:(router.c:624:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni [ 272.090981] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 21:57:25 (1781229445) [ 272.125721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 272.144503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 272.182140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 274.847371] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 21:57:28 (1781229448) [ 275.558491] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_100 Need at least 2 remote nodes found oleg620-server [ 276.267125] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 21:57:30 (1781229450) [ 276.309332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 276.328408] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 276.355423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 276.366752] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 277.057207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp1 --gateway 192.168.206.[21-25]@tcp [ 277.068499] LNet: 30519:0:(router.c:728:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 277.082082] LNetError: 30309:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.206.24@tcp is being used as a gateway but routing feature is not turned on [ 277.088801] LNetError: 30309:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) route to tcp1 through 192.168.206.24@tcp has gone from up to down [ 278.118343] LNetError: 30309:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.206.24@tcp is being used as a gateway but routing feature is not turned on [ 278.124472] LNetError: 30309:0:(router.c:393:lnet_router_discovery_ping_reply()) Skipped 3 previous similar messages [ 279.140722] LNetError: 30309:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.206.25@tcp is being used as a gateway but routing feature is not turned on [ 279.148658] LNetError: 30309:0:(router.c:393:lnet_router_discovery_ping_reply()) Skipped 1 previous similar message [ 279.739987] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 21:57:33 (1781229453) [ 280.160346] LNetError: 30314:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) route to tcp1 through 192.168.206.22@tcp has gone from up to down [ 280.167103] LNetError: 30314:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) Skipped 1 previous similar message [ 280.476849] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_102 Need at least 2 remote nodes found oleg620-server [ 281.158120] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 21:57:35 (1781229455) [ 281.191132] LNetError: 30309:0:(router.c:393:lnet_router_discovery_ping_reply()) Peer 192.168.206.25@tcp is being used as a gateway but routing feature is not turned on [ 281.200283] LNetError: 30309:0:(router.c:393:lnet_router_discovery_ping_reply()) Skipped 3 previous similar messages [ 281.223825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 282.208529] LNetError: 30308:0:(router.c:1262:lnet_check_routers()) Failed to discover router 192.168.206.24@tcp [ 283.297400] LNetError: 31453:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 284.327259] LNet: Removed LNI 192.168.206.20@tcp [ 284.335677] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 284.359413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 284.370827] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 284.650745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route add --net tcp103 --gateway 192.168.206.[21-26/2]@tcp [ 284.662482] LNet: 31571:0:(router.c:728:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled [ 284.669532] LNet: 31571:0:(router.c:728:lnet_add_route()) Skipped 4 previous similar messages [ 284.679374] LNetError: 31458:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) route to tcp103 through 192.168.206.25@tcp has gone from up to down [ 284.680687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl route del --net tcp103 --gateway 192.168.206.[21-26/2]@tcp [ 284.687017] LNetError: 31458:0:(lib-lnet.h:1266:lnet_set_route_aliveness()) Skipped 2 previous similar messages [ 287.262934] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 21:57:41 (1781229461) [ 287.357628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 287.373981] LNetError: 32162:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 287.384657] LNet: Removed LNI 192.168.206.20@tcp [ 287.393781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 287.429611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking -1 [ 287.453701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 287.470953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 287.500387] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 287.530754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 287.549680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 287.573605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 0 [ 287.608337] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 287.627597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 287.665116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 287.703125] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 287.721840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 287.746824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 1 [ 287.792743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 287.812783] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 287.846605] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 287.887463] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 287.906106] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 287.938147] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 2 [ 287.977077] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 288.000586] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 288.030792] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 288.065782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 288.084732] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 288.112182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 288.149502] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 288.169326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 288.198734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 288.236742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 288.253880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 288.280546] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 4 [ 288.310366] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 288.326594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 288.368423] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 291.021566] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 21:57:44 (1781229464) [ 291.854871] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_105 Need at least 2 remote nodes found oleg620-server [ 292.578166] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 21:57:46 (1781229466) [ 293.336347] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_106 Need at least 2 remote nodes found oleg620-server [ 294.169989] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 21:57:47 (1781229467) [ 294.522726] Key type .llcrypt unregistered [ 294.524569] Key type ._llcrypt unregistered [ 296.865786] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 296.938616] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 296.941347] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 296.943901] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 299.172816] Key type ._llcrypt registered [ 299.174542] Key type .llcrypt registered [ 299.271539] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 299.281283] alg: No test for adler32 (adler32-zlib) [ 300.109801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 300.135409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 300.146358] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 300.437455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 300.447622] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 301.273511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 301.284665] LNet: Removed LNI 10.1.2.3@tcp [ 301.332677] LNet: 34282:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 301.335863] LNetError: 34282:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 301.346635] LNet: Removed LNI 192.168.206.20@tcp [ 301.604511] Key type .llcrypt unregistered [ 301.606542] Key type ._llcrypt unregistered [ 306.177817] Key type ._llcrypt registered [ 306.179329] Key type .llcrypt registered [ 306.748663] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 21:58:00 (1781229480) [ 306.979471] Key type .llcrypt unregistered [ 306.980886] Key type ._llcrypt unregistered [ 309.166124] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 309.229327] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 309.231535] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 309.234052] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 311.447050] Key type ._llcrypt registered [ 311.448342] Key type .llcrypt registered [ 311.523252] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 311.530270] alg: No test for adler32 (adler32-zlib) [ 312.340433] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 312.369711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 312.385964] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 312.712891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 312.726031] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 313.765138] LNet: 35773:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 313.768436] LNetError: 35773:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 313.771938] LNet: Removed LNI 192.168.206.20@tcp [ 313.784073] LNet: Removed LNI 10.1.2.3@tcp [ 314.042492] Key type .llcrypt unregistered [ 314.044261] Key type ._llcrypt unregistered [ 318.968085] Key type ._llcrypt registered [ 318.969614] Key type .llcrypt registered [ 319.599310] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 21:58:13 (1781229493) [ 319.817358] Key type .llcrypt unregistered [ 319.819170] Key type ._llcrypt unregistered [ 322.356812] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 322.447658] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 322.450157] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 322.453081] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 324.695653] Key type ._llcrypt registered [ 324.696721] Key type .llcrypt registered [ 324.782259] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 324.791607] alg: No test for adler32 (adler32-zlib) [ 325.623815] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 325.651270] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 325.667891] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 326.601136] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 326.612967] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 327.686060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg [ 327.701738] LNet: Removed LNI 10.1.2.3@tcp [ 327.713716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if test1pg:0 [ 327.738757] LNet: Removed LNI 10.1.2.31@tcp [ 327.750081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 327.766507] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 328.118939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg:0 [ 328.130809] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180] [ 328.591491] LNet: 37690:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 328.595511] LNetError: 37690:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 328.600613] LNet: Removed LNI 10.1.2.3@tcp [ 328.609682] LNet: Removed LNI 10.1.2.31@tcp [ 328.870867] Key type .llcrypt unregistered [ 328.872981] Key type ._llcrypt unregistered [ 333.447163] Key type ._llcrypt registered [ 333.448714] Key type .llcrypt registered [ 334.158595] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 21:58:27 (1781229507) [ 334.325987] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 334.332738] alg: No test for adler32 (adler32-zlib) [ 335.148951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 335.176682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 335.191444] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 335.532770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 335.541700] LNet: 38642:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 335.544679] LNetError: 38642:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 335.552459] LNet: Removed LNI 192.168.206.20@tcp [ 335.562141] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 335.588391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --nid 192.168.206.20@tcp [ 335.602723] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 335.961975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --nid 192.168.206.20@tcp [ 335.985559] LNet: Removed LNI 192.168.206.20@tcp [ 336.281627] Key type .llcrypt unregistered [ 336.283832] Key type ._llcrypt unregistered [ 341.860867] Key type ._llcrypt registered [ 341.862202] Key type .llcrypt registered [ 342.503134] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 21:58:36 (1781229516) [ 343.492398] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_111 Need at least 2 remote nodes found oleg620-server [ 344.348554] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 21:58:38 (1781229518) [ 344.582437] Key type .llcrypt unregistered [ 344.583787] Key type ._llcrypt unregistered [ 347.776870] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 347.859242] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 347.863042] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 347.868310] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 350.186398] Key type ._llcrypt registered [ 350.189614] Key type .llcrypt registered [ 350.271190] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 350.284030] alg: No test for adler32 (adler32-zlib) [ 351.154161] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 351.195986] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 351.631578] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180] [ 352.634953] LNet: 40612:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 352.638746] LNetError: 40612:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 352.642194] LNet: Removed LNI 192.168.206.20@tcp1 [ 352.649643] LNet: Removed LNI 10.1.2.3@tcp2 [ 352.951323] Key type .llcrypt unregistered [ 352.952755] Key type ._llcrypt unregistered [ 357.924649] Key type ._llcrypt registered [ 357.926448] Key type .llcrypt registered [ 358.527685] Lustre: DEBUG MARKER: == sanity-lnet test 150: Check import of ip2nets yaml sequence ========================================================== 21:58:52 (1781229532) [ 358.695361] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 358.702911] alg: No test for adler32 (adler32-zlib) [ 359.532529] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 359.561199] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 359.573790] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 360.022631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 360.034909] LNet: 41580:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 360.039576] LNetError: 41580:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 360.054272] LNet: Removed LNI 192.168.206.20@tcp [ 360.064319] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 360.107548] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet.yaml [ 360.122451] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 363.445398] Lustre: DEBUG MARKER: == sanity-lnet test 151: Check correct application of multiple ip2nets rules ========================================================== 21:58:57 (1781229537) [ 363.494662] LNet: 42163:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 363.498839] LNetError: 42163:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 363.507633] LNet: Removed LNI 192.168.206.20@tcp [ 363.785592] Key type .llcrypt unregistered [ 363.787061] Key type ._llcrypt unregistered [ 366.232333] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 366.319638] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 366.321604] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 366.325288] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 368.574639] Key type ._llcrypt registered [ 368.576408] Key type .llcrypt registered [ 368.668472] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 368.675304] alg: No test for adler32 (adler32-zlib) [ 369.529495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 369.562674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 369.580591] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 370.058499] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 370.074580] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 371.211307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 371.225355] LNet: 42913:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 371.231346] LNetError: 42913:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 371.237778] LNet: Removed LNI 192.168.206.20@tcp [ 371.246732] LNet: Removed LNI 10.1.2.3@tcp [ 371.259224] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 371.304615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 371.324235] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 371.328328] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 371.579505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 371.600623] LNet: Removed LNI 192.168.206.20@tcp [ 371.616928] LNet: Removed LNI 10.1.2.3@tcp [ 371.633622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 371.683856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 371.701818] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 371.705080] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 371.907530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 371.924574] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 372.532869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 372.553192] LNet: 43102:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 372.557352] LNet: 43102:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 372.562954] LNetError: 43102:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 372.567539] LNetError: 43102:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 372.572331] LNet: Removed LNI 192.168.206.20@tcp [ 372.574606] LNet: Removed LNI 10.1.2.3@tcp [ 372.583153] LNet: Removed LNI 192.168.206.20@tcp2 [ 372.604357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 372.658587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-151.yaml [ 372.680069] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 372.684117] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 372.687596] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 376.248212] Lustre: DEBUG MARKER: == sanity-lnet test 152: Check import of ip2nets with wildcard IP patterns ========================================================== 21:59:09 (1781229549) [ 376.305756] LNet: 43692:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 376.312050] LNetError: 43692:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 376.317399] LNetError: 43692:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 376.322755] LNet: Removed LNI 192.168.206.20@tcp [ 376.324272] LNet: Removed LNI 10.1.2.3@tcp [ 376.331630] LNet: Removed LNI 192.168.206.20@tcp2 [ 376.616784] Key type .llcrypt unregistered [ 376.619124] Key type ._llcrypt unregistered [ 378.917750] Key type ._llcrypt registered [ 378.919263] Key type .llcrypt registered [ 379.000555] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 379.007825] alg: No test for adler32 (adler32-zlib) [ 379.823817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 379.854364] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 379.868692] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 380.255955] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 380.266428] LNet: 44092:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 380.270502] LNetError: 44092:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 380.281357] LNet: Removed LNI 192.168.206.20@tcp [ 380.292447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 380.320424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 380.334261] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 380.530225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 380.548469] LNet: Removed LNI 192.168.206.20@tcp [ 380.557911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 380.587717] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 380.601904] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 380.789186] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 380.799415] LNet: 44152:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 380.803065] LNet: 44152:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 380.807100] LNetError: 44152:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 380.811047] LNetError: 44152:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 380.825912] LNet: Removed LNI 192.168.206.20@tcp [ 380.835534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 380.863590] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 380.874804] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 381.064524] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 381.101076] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 381.103439] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 381.106311] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 383.291405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 383.301215] LNet: 44231:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 383.306133] LNetError: 44231:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 383.319408] LNet: Removed LNI 192.168.206.20@tcp [ 383.328781] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 383.358177] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-152.yaml [ 383.370356] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 385.937752] Lustre: DEBUG MARKER: == sanity-lnet test 153: Check import of ip2nets with IP address ranges ========================================================== 21:59:19 (1781229559) [ 385.975606] LNet: 44821:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 385.979964] LNetError: 44821:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 385.991593] LNet: Removed LNI 192.168.206.20@tcp [ 386.244877] Key type .llcrypt unregistered [ 386.246383] Key type ._llcrypt unregistered [ 388.304250] Key type ._llcrypt registered [ 388.305660] Key type .llcrypt registered [ 388.373395] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 388.379972] alg: No test for adler32 (adler32-zlib) [ 389.184201] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 389.206827] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 389.218046] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 389.534129] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 389.543416] LNet: 45225:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 389.547480] LNetError: 45225:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 389.558521] LNet: Removed LNI 192.168.206.20@tcp [ 389.567419] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 389.593994] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-153.yaml [ 389.605775] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 392.041107] Lustre: DEBUG MARKER: == sanity-lnet test 154: Check import of ip2nets with stepped IP address ranges ========================================================== 21:59:25 (1781229565) [ 392.086263] LNet: 45809:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 392.089735] LNetError: 45809:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 392.097607] LNet: Removed LNI 192.168.206.20@tcp [ 392.357599] Key type .llcrypt unregistered [ 392.359192] Key type ._llcrypt unregistered [ 394.598171] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 394.675452] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 394.678322] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 394.681867] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 396.901730] Key type ._llcrypt registered [ 396.903983] Key type .llcrypt registered [ 396.989919] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 396.997648] alg: No test for adler32 (adler32-zlib) [ 397.826301] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 397.858658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 397.869314] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 398.296443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 398.306723] LNet: 46264:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 398.310399] LNetError: 46264:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 398.322500] LNet: Removed LNI 192.168.206.20@tcp [ 398.335935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 398.371491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-154.yaml [ 398.388668] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 401.325856] Lustre: DEBUG MARKER: == sanity-lnet test 155: Check ip2nets import failure with no matching rules ========================================================== 21:59:35 (1781229575) [ 401.366305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 401.378403] LNet: 46854:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 401.382189] LNetError: 46854:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 401.392705] LNet: Removed LNI 192.168.206.20@tcp [ 401.405584] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 401.460935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-155.yaml [ 404.454553] Lustre: DEBUG MARKER: == sanity-lnet test 156: Check import of ip2nets with comma-separated IP ranges ========================================================== 21:59:38 (1781229578) [ 404.494346] LNet: 47435:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 404.797872] Key type .llcrypt unregistered [ 404.800344] Key type ._llcrypt unregistered [ 407.179739] Key type ._llcrypt registered [ 407.181049] Key type .llcrypt registered [ 407.252375] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 407.258622] alg: No test for adler32 (adler32-zlib) [ 408.074133] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 408.101756] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 408.112845] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 408.464034] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 408.472751] LNet: 47840:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 408.475808] LNetError: 47840:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 408.484372] LNet: Removed LNI 192.168.206.20@tcp [ 408.493137] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 408.518533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-156.yaml [ 408.530845] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 411.227669] Lustre: DEBUG MARKER: == sanity-lnet test 157: Check import failure with malformed ip2nets YAML ========================================================== 21:59:45 (1781229585) [ 411.272746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 411.285851] LNet: 48429:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 411.289971] LNetError: 48429:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 411.299869] LNet: Removed LNI 192.168.206.20@tcp [ 411.310927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 411.344438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-157.yaml [ 413.756212] Lustre: DEBUG MARKER: == sanity-lnet test 158: Check import of ip2nets with single IPv6 address ========================================================== 21:59:47 (1781229587) [ 414.338720] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_158 Interface has no IPv6 [ 415.073808] Lustre: DEBUG MARKER: == sanity-lnet test 159: Check import of ip2nets with IPv6 /128 CIDR notation ========================================================== 21:59:48 (1781229588) [ 415.702986] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_159 Interface has no IPv6 [ 416.373362] Lustre: DEBUG MARKER: == sanity-lnet test 160: Check correct application of multiple IPv6 ip2nets rules ========================================================== 21:59:50 (1781229590) [ 417.072142] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_160 Interface has no IPv6 [ 417.822327] Lustre: DEBUG MARKER: == sanity-lnet test 161: Check import of ip2nets with various IPv6 prefix lengths ========================================================== 21:59:51 (1781229591) [ 418.405551] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_161 Interface has no IPv6 [ 419.042296] Lustre: DEBUG MARKER: == sanity-lnet test 162: Check ip2nets import failure with non-matching IPv6 ========================================================== 21:59:52 (1781229592) [ 419.663643] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_162 Interface has no IPv6 [ 420.336958] Lustre: DEBUG MARKER: == sanity-lnet test 170: Check CPTs and tunables parsing when importing ========================================================== 21:59:54 (1781229594) [ 420.377659] LNet: 50472:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 420.648523] Key type .llcrypt unregistered [ 420.650070] Key type ._llcrypt unregistered [ 422.916604] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 422.950143] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 422.952879] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 422.956030] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 425.195308] Key type ._llcrypt registered [ 425.196663] Key type .llcrypt registered [ 425.271877] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 425.280298] alg: No test for adler32 (adler32-zlib) [ 426.094363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 426.131735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 426.142195] LNet: Added LNI 192.168.206.20@tcp [8/512/0/180] [ 426.145048] LNet: Added LNI 10.1.2.3@tcp [8/512/0/180] [ 426.146382] LNet: 50800:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 426.186878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 426.195523] LNet: 50819:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 426.198184] LNetError: 50819:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 426.201762] LNet: Removed LNI 192.168.206.20@tcp [ 426.203025] LNetError: 50819:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 426.212327] LNet: Removed LNI 10.1.2.3@tcp [ 426.221756] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 426.250979] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 426.261744] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 426.287891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 426.296458] LNet: 50843:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 426.299498] LNetError: 50843:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 426.312587] LNet: Removed LNI 192.168.206.20@tcp [ 426.323325] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 426.353140] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 426.364615] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 426.367723] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 426.369738] LNet: 50851:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 426.413612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 426.424263] LNet: 50870:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 426.427406] LNetError: 50870:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 426.431339] LNet: Removed LNI 192.168.206.20@tcp [ 426.433316] LNetError: 50870:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 426.444610] LNet: Removed LNI 10.1.2.3@tcp [ 426.453721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 426.482191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 426.496157] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 426.499921] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 426.501649] LNet: 50879:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 426.583833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 426.593949] LNet: 50905:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 426.597859] LNetError: 50905:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 426.601803] LNet: Removed LNI 192.168.206.20@tcp [ 426.603789] LNetError: 50905:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 426.613315] LNet: Removed LNI 10.1.2.3@tcp [ 426.622533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 426.651367] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 426.661438] LNet: Added LNI 192.168.206.20@tcp [8/256/0/182] [ 426.664341] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 426.703264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 426.714994] LNet: 50933:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 426.719180] LNetError: 50933:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 426.722810] LNet: Removed LNI 192.168.206.20@tcp [ 426.724138] LNetError: 50933:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 426.735331] LNet: Removed LNI 10.1.2.3@tcp [ 426.745165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 426.777455] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --old-api [ 426.790741] LNet: Added LNI 192.168.206.20@tcp [8/256/0/182] [ 426.792900] LNet: Added LNI 10.1.2.3@tcp [8/256/0/182] [ 426.795173] LNet: 50941:0:(api-ni.c:2864:lnet_startup_lndnet()) All tunables of NIs of a net should be the same [ 426.831104] LNet: Removed LNI 192.168.206.20@tcp [ 426.847127] LNet: Removed LNI 10.1.2.3@tcp [ 427.109847] Key type .llcrypt unregistered [ 427.111547] Key type ._llcrypt unregistered [ 431.358841] Key type ._llcrypt registered [ 431.360110] Key type .llcrypt registered [ 431.921592] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 22:00:05 (1781229605) [ 432.099741] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 432.106691] alg: No test for adler32 (adler32-zlib) [ 432.917630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 432.943698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 432.958049] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 433.270740] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --ip2net tcp 192.168.206.20 [ 433.280476] LNet: 51915:0:(config.c:1627:lnet_parse_ip2nets()) Kernel parsing of ip2nets is deprecated. Consider converting to LNet YAML configuration. [ 433.284917] LNet: Removed LNI 192.168.206.20@tcp [ 435.714360] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 22:00:09 (1781229609) [ 435.762330] LNet: 52477:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 435.768464] LNetError: 52477:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 435.783088] LNet: Removed LNI 192.168.206.20@tcp [ 436.044596] Key type .llcrypt unregistered [ 436.046484] Key type ._llcrypt unregistered [ 438.595519] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 438.716544] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 441.105694] Key type ._llcrypt registered [ 441.108070] Key type .llcrypt registered [ 441.190285] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 441.199073] alg: No test for adler32 (adler32-zlib) [ 442.264922] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 445.197630] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 22:00:19 (1781229619) [ 445.252360] LNet: 53406:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 445.255663] LNetError: 53406:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 445.266677] LNet: Removed LNI 10.1.2.3@tcp [ 445.585979] Key type .llcrypt unregistered [ 445.588106] Key type ._llcrypt unregistered [ 448.161933] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 448.309534] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 448.312384] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 448.316699] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 450.714077] Key type ._llcrypt registered [ 450.715695] Key type .llcrypt registered [ 450.792531] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 450.798429] alg: No test for adler32 (adler32-zlib) [ 451.640663] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 454.472634] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 22:00:28 (1781229628) [ 455.105296] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_202 kernel ip2nets deprecated in 2.17.53 [ 455.770314] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 22:00:29 (1781229629) [ 455.809360] LNet: 54586:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 455.815352] LNetError: 54586:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 455.827550] LNet: Removed LNI 10.1.2.3@tcp [ 456.079679] Key type .llcrypt unregistered [ 456.081870] Key type ._llcrypt unregistered [ 458.379656] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 458.499947] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 458.502474] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 458.504668] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 460.853056] Key type ._llcrypt registered [ 460.854404] Key type .llcrypt registered [ 460.951208] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 460.960461] alg: No test for adler32 (adler32-zlib) [ 461.794670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 461.839221] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 465.807533] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 22:00:39 (1781229639) [ 466.027736] LNet: 55836:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 466.031747] LNetError: 55836:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 466.046652] LNet: Removed LNI 10.1.2.3@tcp [ 466.310559] Key type .llcrypt unregistered [ 466.311980] Key type ._llcrypt unregistered [ 468.943272] Key type ._llcrypt registered [ 468.945120] Key type .llcrypt registered [ 469.059133] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 469.070441] alg: No test for adler32 (adler32-zlib) [ 469.922662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 469.947875] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 473.435775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 473.783840] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_interrupt -r 1 [ 473.809172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_interrupt -r 1 [ 473.837587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 473.853769] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 473.986191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 474.011297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 474.033974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 474.056678] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 474.081714] LNet: 1 local NIs in recovery (showing 1): 192.168.206.20@tcp [ 474.085422] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 474.115321] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 474.134547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 474.158187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 474.172268] LNet: 56273:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 474.187671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 474.212674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 474.243885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 474.277608] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 474.308953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 474.330364] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 474.368384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 474.681704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_dropped -r 1 [ 474.704413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_dropped -r 1 [ 474.739350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 474.757471] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 474.763206] LNet: Skipped 1 previous similar message [ 474.904696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 474.926541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 474.946354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 474.965735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 474.992459] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 475.023511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 475.055636] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 475.085654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 475.095074] LNet: 56382:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 475.106284] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 475.124735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 475.149922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 475.177946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 475.203703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 475.225280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 475.249743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 475.585696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_aborted -r 1 [ 475.608359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_aborted -r 1 [ 475.636326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 475.783991] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 475.800110] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 475.819622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 475.842713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 475.866464] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 475.884803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 475.907518] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 475.926621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 475.946694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 475.966171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 475.994698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 476.018604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 476.041615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 476.067316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 476.085891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 476.410117] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_no_route -r 1 [ 476.435315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_no_route -r 1 [ 476.454530] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 476.467796] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 476.471568] LNet: Skipped 1 previous similar message [ 476.587919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 476.605961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 476.622734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 476.640658] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 476.659985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 476.678708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 476.697788] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 476.715110] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 476.728741] LNet: 56600:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 476.735139] LNet: 56600:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 476.751127] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 476.769812] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 476.790263] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 476.812156] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 476.829461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 476.852288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 476.877334] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 477.151779] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_timeout -r 1 [ 477.173116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_timeout -r 1 [ 477.195305] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 477.312648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 477.333992] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 477.352560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 477.379771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 477.405746] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 477.427441] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 477.447181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 477.470630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 477.496648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 477.515308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 477.533817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 477.550449] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 477.567175] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 477.586927] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 477.604217] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 477.823438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_error -r 1 [ 477.842830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_error -r 1 [ 477.866447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 477.983215] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 478.002117] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 478.021539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 478.041204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 478.060055] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 478.075829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 478.090550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 478.106662] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 478.128466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 478.148188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 478.170431] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 478.192589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 478.211384] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 478.231534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 478.252719] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 478.397122] LNet: 56860:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 478.400678] LNetError: 56860:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 478.409396] LNet: Removed LNI 192.168.206.20@tcp [ 478.606820] Key type .llcrypt unregistered [ 478.608778] Key type ._llcrypt unregistered [ 485.308502] Key type ._llcrypt registered [ 485.309958] Key type .llcrypt registered [ 485.901709] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 22:00:59 (1781229659) [ 486.271394] Key type .llcrypt unregistered [ 486.272902] Key type ._llcrypt unregistered [ 488.925231] Key type ._llcrypt registered [ 488.926796] Key type .llcrypt registered [ 489.011899] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 489.019661] alg: No test for adler32 (adler32-zlib) [ 489.875722] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 489.904828] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 493.824660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 498.547947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 498.566593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 498.579058] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 499.205671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_interrupt -r 1 [ 499.224251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e local_interrupt -r 1 [ 499.243690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_interrupt -r 1 [ 499.266698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e local_interrupt -r 1 [ 499.288626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e local_interrupt -r 1 [ 499.307657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e local_interrupt -r 1 [ 499.330593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e local_interrupt -r 1 [ 499.358123] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e local_interrupt -r 1 [ 499.375779] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 499.385345] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 499.387955] LNet: 1 local NIs in recovery (showing 1): 192.168.206.20@tcp [ 499.497541] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 499.511614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 499.527760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 499.548024] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 499.566360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 499.581509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 499.600484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 499.623477] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 499.632483] LNet: 58336:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 499.641122] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 499.658061] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 499.676833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 499.695803] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 499.714370] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 499.738936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 499.760577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 500.016871] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_dropped -r 1 [ 500.035785] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e local_dropped -r 1 [ 500.055176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_dropped -r 1 [ 500.085957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e local_dropped -r 1 [ 500.107339] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e local_dropped -r 1 [ 500.130188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e local_dropped -r 1 [ 500.159922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e local_dropped -r 1 [ 500.186614] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e local_dropped -r 1 [ 500.206485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 500.216505] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 500.220040] LNet: Skipped 2 previous similar messages [ 500.317050] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 500.332814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 500.348258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 500.363071] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 500.384046] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 500.404692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 500.425451] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 500.445058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 500.454621] LNet: 58457:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 500.464126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 500.479438] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 500.494789] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 500.515435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 500.533892] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 500.550800] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 500.567981] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 500.808769] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_aborted -r 1 [ 500.830235] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e local_aborted -r 1 [ 500.849699] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_aborted -r 1 [ 500.866957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e local_aborted -r 1 [ 500.889523] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e local_aborted -r 1 [ 500.907255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e local_aborted -r 1 [ 500.925298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e local_aborted -r 1 [ 500.944576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e local_aborted -r 1 [ 500.960831] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 501.075884] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 501.097201] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 501.115061] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 501.129754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 501.149406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 501.167556] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 501.184943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 501.204461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 501.219116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 501.236728] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 501.251777] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 501.266534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 501.283158] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 501.301778] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 501.323429] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 501.564074] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_no_route -r 1 [ 501.596210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e local_no_route -r 1 [ 501.618399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_no_route -r 1 [ 501.646868] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e local_no_route -r 1 [ 501.668273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e local_no_route -r 1 [ 501.688467] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e local_no_route -r 1 [ 501.708625] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e local_no_route -r 1 [ 501.725855] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e local_no_route -r 1 [ 501.743935] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 501.753684] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 501.757761] LNet: Skipped 6 previous similar messages [ 501.859267] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 501.879487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 501.899499] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 501.918288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 501.937170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 501.954850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 501.971169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 501.991252] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 502.001901] LNet: 58699:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 502.007643] LNet: 58699:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 502.026207] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 502.043849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 502.064472] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 502.082359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 502.105826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 502.134332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 502.162825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 502.381189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_timeout -r 1 [ 502.399373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e local_timeout -r 1 [ 502.417816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_timeout -r 1 [ 502.435597] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e local_timeout -r 1 [ 502.460631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e local_timeout -r 1 [ 502.483157] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e local_timeout -r 1 [ 502.505273] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e local_timeout -r 1 [ 502.526792] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e local_timeout -r 1 [ 502.546462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 502.669445] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 502.687670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 502.703272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 502.718880] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 502.734981] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 502.754849] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 502.773634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 502.794112] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 502.816241] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 502.834882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 502.855349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 502.870274] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 502.890356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 502.907497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 502.928755] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 503.142776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_error -r 1 [ 503.162235] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e local_error -r 1 [ 503.181654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_error -r 1 [ 503.200485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e local_error -r 1 [ 503.226670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e local_error -r 1 [ 503.251975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e local_error -r 1 [ 503.274575] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e local_error -r 1 [ 503.301714] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e local_error -r 1 [ 503.322264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 503.424896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 503.440537] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 503.454708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 503.470103] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 503.485060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 503.499667] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 503.515376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 503.532925] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 503.549353] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 503.565696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 503.579716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 503.606887] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 503.630246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 503.650118] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 503.670214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 506.429919] LNet: 59017:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 506.434194] LNetError: 59017:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 506.438330] LNet: Removed LNI 192.168.206.20@tcp [ 507.494523] LNet: Removed LNI 192.168.206.20@tcp1 [ 507.662660] Key type .llcrypt unregistered [ 507.664514] Key type ._llcrypt unregistered [ 513.217695] Key type ._llcrypt registered [ 513.218986] Key type .llcrypt registered [ 513.683479] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 22:01:27 (1781229687) [ 513.983538] Key type .llcrypt unregistered [ 513.985196] Key type ._llcrypt unregistered [ 516.307053] Key type ._llcrypt registered [ 516.308548] Key type .llcrypt registered [ 516.394300] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 516.404980] alg: No test for adler32 (adler32-zlib) [ 517.224821] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 517.241962] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 520.338611] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 520.662881] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e remote_dropped -r 1 [ 520.681338] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e remote_dropped -r 1 [ 520.697283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 520.708163] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 520.817540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 520.832941] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 520.848191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 520.867724] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 520.888413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 520.916352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 520.932484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 520.950462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 520.964341] LNet: 60270:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 520.978768] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 521.001891] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 521.020228] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 521.041948] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 521.063243] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 521.081480] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 521.101890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 521.373212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e remote_error -r 1 [ 521.393206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e remote_error -r 1 [ 521.418386] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 521.428892] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 521.533052] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 521.553787] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 521.574910] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 521.591100] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 521.606487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 521.621302] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 521.636633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 521.655653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 521.667723] LNet: 60379:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 521.679130] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 521.697980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 521.715816] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 521.731297] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 521.747466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 521.766668] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 521.783522] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 522.027923] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e remote_timeout -r 1 [ 522.044600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e remote_timeout -r 1 [ 522.063564] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 522.166903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 522.182736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 522.199470] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 522.215332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 522.233600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 522.253019] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 522.270750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 522.299203] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 522.315132] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 522.332680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 522.347180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 522.368069] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 522.389093] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 522.407051] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 522.424996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 522.545338] LNet: 60529:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 522.549292] LNetError: 60529:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 523.621925] LNet: Removed LNI 192.168.206.20@tcp [ 523.797434] Key type .llcrypt unregistered [ 523.799218] Key type ._llcrypt unregistered [ 529.980180] Key type ._llcrypt registered [ 529.981967] Key type .llcrypt registered [ 530.582549] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 22:01:44 (1781229704) [ 531.052946] Key type .llcrypt unregistered [ 531.054512] Key type ._llcrypt unregistered [ 533.395697] Key type ._llcrypt registered [ 533.397440] Key type .llcrypt registered [ 533.475675] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 533.482685] alg: No test for adler32 (adler32-zlib) [ 534.294281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 534.309218] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 537.702550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 540.961206] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 540.979736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 540.990789] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 541.610885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e remote_dropped -r 1 [ 541.632628] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e remote_dropped -r 1 [ 541.652525] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e remote_dropped -r 1 [ 541.671591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e remote_dropped -r 1 [ 541.688547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e remote_dropped -r 1 [ 541.710563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e remote_dropped -r 1 [ 541.729316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e remote_dropped -r 1 [ 541.749212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e remote_dropped -r 1 [ 541.769822] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 541.781231] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 541.784172] LNet: 1 peer NIs in recovery (showing 1): 192.168.206.120@tcp [ 541.874521] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 541.889447] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 541.904032] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 541.917589] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 541.930413] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 541.946124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 541.961341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 541.976258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 541.986226] LNet: 61991:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 541.995775] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 542.012238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 542.028069] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 542.042558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 542.059485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 542.074782] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 542.088902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 542.104033] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 542.287600] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e remote_error -r 1 [ 542.306648] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e remote_error -r 1 [ 542.325244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e remote_error -r 1 [ 542.345776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e remote_error -r 1 [ 542.364437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e remote_error -r 1 [ 542.382152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e remote_error -r 1 [ 542.399192] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e remote_error -r 1 [ 542.418183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e remote_error -r 1 [ 542.435341] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 542.445187] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 542.448180] LNet: Skipped 2 previous similar messages [ 542.525818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 542.539460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 542.555238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 542.570943] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 542.586917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 542.601799] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 542.616869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 542.632686] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 542.642336] LNet: 62114:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 542.652081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 542.667395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 542.681581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 542.697846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 542.714342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 542.732414] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 542.747879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 542.763547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 542.961258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e remote_timeout -r 1 [ 542.979430] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e remote_timeout -r 1 [ 542.999630] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e remote_timeout -r 1 [ 543.017690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e remote_timeout -r 1 [ 543.036559] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e remote_timeout -r 1 [ 543.054466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e remote_timeout -r 1 [ 543.075264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e remote_timeout -r 1 [ 543.092985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e remote_timeout -r 1 [ 543.111921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.120@tcp [ 543.202671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 543.218865] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 543.233836] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 543.250126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 543.268642] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 543.287474] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 543.303960] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 543.320441] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 543.336424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 543.351634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 543.367403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 543.383072] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 543.403370] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 543.419422] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 543.435364] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 543.451666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 545.884582] LNet: 62315:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 545.888578] LNetError: 62315:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 545.892539] LNet: Removed LNI 192.168.206.20@tcp [ 545.902207] LNet: Removed LNI 192.168.206.20@tcp1 [ 546.072660] Key type .llcrypt unregistered [ 546.074350] Key type ._llcrypt unregistered [ 552.221596] Key type ._llcrypt registered [ 552.223174] Key type .llcrypt registered [ 552.836222] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 22:02:06 (1781229726) [ 553.479535] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_208 kernel ip2nets deprecated in 2.17.53 [ 554.133140] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 22:02:08 (1781229728) [ 554.468428] Key type .llcrypt unregistered [ 554.470168] Key type ._llcrypt unregistered [ 556.926296] Key type ._llcrypt registered [ 556.928023] Key type .llcrypt registered [ 557.005516] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 557.012180] alg: No test for adler32 (adler32-zlib) [ 557.826596] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 557.838574] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 561.043360] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 561.330435] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e network_timeout -r 1 [ 561.349588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e network_timeout -r 1 [ 561.369712] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 561.381437] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 561.471372] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 561.485975] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 561.502202] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 561.521058] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 561.537751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 561.550983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 561.565417] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 561.583534] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 561.594993] LNet: 63861:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 561.604784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 561.620763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 561.638968] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 561.654683] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 561.672543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 561.692498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 561.818866] LNet: 63900:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 561.822369] LNetError: 63900:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 561.838402] LNet: Removed LNI 192.168.206.20@tcp [ 562.012152] Key type .llcrypt unregistered [ 562.013868] Key type ._llcrypt unregistered [ 568.387405] Key type ._llcrypt registered [ 568.388959] Key type .llcrypt registered [ 568.460162] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 568.466266] alg: No test for adler32 (adler32-zlib) [ 569.281882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 569.298577] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 572.609776] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 575.806663] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 575.827099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 575.840701] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 576.495122] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e network_timeout -r 1 [ 576.517162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e network_timeout -r 1 [ 576.540027] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e network_timeout -r 1 [ 576.562974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e network_timeout -r 1 [ 576.585497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e network_timeout -r 1 [ 576.612985] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e network_timeout -r 1 [ 576.642646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e network_timeout -r 1 [ 576.664152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e network_timeout -r 1 [ 576.682704] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 576.693632] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 576.784749] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 576.803570] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 576.821708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 576.838843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 576.858543] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 576.876826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 576.893969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 576.915195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 576.924907] LNet: 64780:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 576.935693] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 576.954120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 576.974686] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 576.997526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 577.019744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 577.037363] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 579.367973] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 579.510418] LNet: 64853:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 579.516279] LNetError: 64853:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 579.523090] LNet: Removed LNI 192.168.206.20@tcp [ 579.532195] LNet: Removed LNI 192.168.206.20@tcp1 [ 579.709560] Key type .llcrypt unregistered [ 579.710908] Key type ._llcrypt unregistered [ 586.177456] Key type ._llcrypt registered [ 586.179335] Key type .llcrypt registered [ 586.806465] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 22:02:40 (1781229760) [ 586.965332] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 586.971521] alg: No test for adler32 (adler32-zlib) [ 587.791911] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 587.820949] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 587.837185] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 588.184914] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 588.200638] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 588.634867] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 588.680888] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 588.710165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 588.740759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 588.763742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 588.791457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 588.817870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.206.20@tcp [ 588.835513] LNetError: 65951:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.206.20@tcp to recovery [ 589.856289] LNet: 1 local NIs in recovery (showing 1): 192.168.206.20@tcp [ 589.856498] LNet: There was an unexpected network error while writing to 192.168.206.20: rc = -22 [ 591.905551] LNet: There was an unexpected network error while writing to 192.168.206.20: rc = -22 [ 592.053351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 596.000383] LNet: There was an unexpected network error while writing to 192.168.206.20: rc = -22 [ 596.321591] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 596.337315] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 596.369351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 596.382074] LNet: 66068:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 596.387469] LNetError: 66068:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 596.391332] LNet: Removed LNI 192.168.206.20@tcp [ 596.397810] LNet: Removed LNI 192.168.206.20@tcp1 [ 596.407780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 596.434801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 596.446748] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 596.790151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 596.805948] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 597.251833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 597.273182] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 597.301633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 597.323268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e local_error [ 597.342621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e local_error [ 597.359306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 597.374407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 597.389450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.206.20@tcp [ 597.400942] LNetError: 66340:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.206.20@tcp to recovery [ 599.520234] LNet: There was an unexpected network error while writing to 192.168.206.20: rc = -22 [ 599.522507] LNet: Skipped 1 previous similar message [ 600.577765] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 603.616368] LNet: There was an unexpected network error while writing to 192.168.206.20: rc = -22 [ 607.895609] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 607.913697] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 607.930817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 610.439352] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 22:03:04 (1781229784) [ 610.484205] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 610.497751] LNet: 67054:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 610.501535] LNetError: 67054:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 610.504932] LNetError: 67054:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 610.508198] LNet: Removed LNI 192.168.206.20@tcp [ 610.515666] LNet: Removed LNI 192.168.206.20@tcp1 [ 610.524051] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 610.547846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 610.560667] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 610.861383] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 610.870939] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 611.236146] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 611.262929] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 10 [ 611.276307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 611.292519] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 611.308353] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 611.324355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -r 1 [ 611.341079] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.206.20@tcp --health 0 [ 611.383919] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 611.552166] LNet: 1 peer NIs in recovery (showing 1): 192.168.206.20@tcp [ 612.576633] LNet: There was an unexpected network error while writing to 192.168.206.20: rc = -22 [ 612.580218] LNet: Skipped 1 previous similar message [ 626.964251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 627.037635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 627.050986] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 627.068493] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.206.20@tcp --health 500 [ 627.115434] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 634.377996] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 634.402522] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 634.413323] LNet: 67564:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 634.417165] LNetError: 67564:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 634.421126] LNetError: 67564:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 634.425289] LNet: Removed LNI 192.168.206.20@tcp [ 634.434331] LNet: Removed LNI 192.168.206.20@tcp1 [ 634.444220] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 634.468457] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 634.480718] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 634.792382] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 634.803766] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 635.155171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 635.172400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 635.200055] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 4 [ 635.217808] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -m GET -r 1 -e remote_error [ 635.235869] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp1 -d *@tcp1 -m GET -r 1 -e remote_error [ 635.252928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 635.276511] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp1 -d *@tcp1 -r 1 [ 635.295258] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 192.168.206.20@tcp --health 0 [ 636.513274] LNet: There was an unexpected network error while writing to 192.168.206.20: rc = -22 [ 636.516642] LNet: Skipped 3 previous similar messages [ 637.443660] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 646.896120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 646.911120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 646.924520] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 649.382581] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 22:03:43 (1781229823) [ 649.527532] LNet: 68620:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 649.530453] LNetError: 68620:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 649.535343] LNetError: 68620:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 649.540110] LNet: Removed LNI 192.168.206.20@tcp [ 649.551434] LNet: Removed LNI 192.168.206.20@tcp1 [ 649.798754] Key type .llcrypt unregistered [ 649.800319] Key type ._llcrypt unregistered [ 651.941924] Key type ._llcrypt registered [ 651.943422] Key type .llcrypt registered [ 652.030412] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 652.036821] alg: No test for adler32 (adler32-zlib) [ 652.854751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 652.868158] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 656.651498] Lustre: DEBUG MARKER: Initial discovery [ 656.690692] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.206.120@tcp [ 657.603789] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag [ 657.625946] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 -e local_error [ 657.652833] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.206.120@tcp [ 657.670345] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 657.681244] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 657.696088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay add -s *@tcp -d 192.168.206.120@tcp -r 1 -m GET -l 3 [ 657.713698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.206.120@tcp -r 1 -m GET -e local_error [ 657.738606] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d 192.168.206.120@tcp -r 1 -m PUT -l 6 [ 658.016381] LNet: 1 local NIs in recovery (showing 1): 192.168.206.20@tcp [ 658.349715] Lustre: DEBUG MARKER: Force oleg620-client.virtnet to discover 192.168.206.120@tcp (in background) [ 658.375503] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover --force 192.168.206.120@tcp [ 658.952457] Lustre: DEBUG MARKER: Force oleg620-server to discover 192.168.206.20@tcp [ 659.867688] Lustre: DEBUG MARKER: Wait for 69373 [ 661.408274] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 661.992684] Lustre: DEBUG MARKER: Finished wait on 69373 [ 662.014581] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_delay del -a [ 662.035320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop del -a [ 662.142287] LNet: 69831:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 662.145117] LNetError: 69831:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 662.153594] LNet: Removed LNI 192.168.206.20@tcp [ 662.319465] Key type .llcrypt unregistered [ 662.320849] Key type ._llcrypt unregistered [ 668.281186] Key type ._llcrypt registered [ 668.282897] Key type .llcrypt registered [ 668.867732] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 22:04:02 (1781229842) [ 669.059341] Key type .llcrypt unregistered [ 669.061135] Key type ._llcrypt unregistered [ 671.378495] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 671.469458] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 671.473656] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 671.476837] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 673.721624] Key type ._llcrypt registered [ 673.723356] Key type .llcrypt registered [ 673.811635] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 673.817230] alg: No test for adler32 (adler32-zlib) [ 674.648368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 674.684571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 674.700362] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 675.048183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 675.062197] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 678.720423] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 22:04:12 (1781229852) [ 678.759150] LNet: 71900:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 678.764300] LNetError: 71900:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 678.768276] LNet: Removed LNI 192.168.206.20@tcp [ 678.773726] LNet: Removed LNI 10.1.2.3@tcp [ 679.086163] Key type .llcrypt unregistered [ 679.087616] Key type ._llcrypt unregistered [ 681.704394] Key type ._llcrypt registered [ 681.706229] Key type .llcrypt registered [ 681.815294] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 681.826672] alg: No test for adler32 (adler32-zlib) [ 682.636773] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 682.663332] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 682.676911] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 683.020837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 683.032591] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 683.489882] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 10.1.2.3@tcp 192.168.206.20@tcp [ 684.512140] LNetError: Unexpected error -22 connecting to 192.168.206.20@tcp at host 192.168.206.20:988 [ 686.359282] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 22:04:20 (1781229860) [ 686.420555] LNet: 73032:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 686.424870] LNetError: 73032:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 686.428347] LNet: Removed LNI 192.168.206.20@tcp [ 686.433730] LNet: Removed LNI 10.1.2.3@tcp [ 686.702401] Key type .llcrypt unregistered [ 686.704718] Key type ._llcrypt unregistered [ 689.064376] Key type ._llcrypt registered [ 689.066204] Key type .llcrypt registered [ 689.168064] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 689.177175] alg: No test for adler32 (adler32-zlib) [ 690.003156] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 690.035785] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 690.050038] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 690.381730] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 690.394337] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 690.845947] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 192.168.206.20@tcp1 --nid 192.168.206.20@tcp2 [ 692.752205] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.20@tcp1 192.168.206.20@tcp2 [ 692.764302] LNetError: 73872:0:(lib-move.c:2007:lnet_handle_find_routed_path()) Unable to send message from 192.168.206.20@tcp1 to 192.168.206.20@tcp2 - Route table may be misconfigured [ 692.779641] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.20@tcp2 192.168.206.20@tcp1 [ 695.439364] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 22:04:29 (1781229869) [ 695.475369] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 695.486323] LNet: 74438:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 695.489978] LNetError: 74438:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 695.493474] LNet: Removed LNI 192.168.206.20@tcp1 [ 696.552259] LNet: Removed LNI 192.168.206.20@tcp2 [ 696.565436] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 696.600721] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 696.618546] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 696.949736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 696.963702] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 697.342126] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 697.360162] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.20@tcp 192.168.206.20@tcp [ 697.377426] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.20@tcp1 192.168.206.20@tcp1 [ 697.401342] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.206.20@tcp -d 192.168.206.20@tcp -e network_timeout [ 697.419411] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -e network_timeout [ 697.438842] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -e network_timeout [ 697.457696] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -r 1 -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -e network_timeout [ 697.478318] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 697.490113] LNet: There was an unexpected network error while writing to 192.168.206.20: rc = -22 [ 697.502962] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop del -a [ 697.547290] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -p [ 697.593973] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug recovery -l [ 697.633149] LNet: 1 local NIs in recovery (showing 1): 192.168.206.20@tcp [ 700.198726] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 22:04:34 (1781229874) [ 700.250547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 700.262672] LNet: 75303:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 700.265714] LNetError: 75303:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 700.268635] LNetError: 75303:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 700.272524] LNet: Removed LNI 192.168.206.20@tcp [ 701.288072] LNet: Removed LNI 192.168.206.20@tcp1 [ 701.298694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 701.342536] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 0@lo [ 701.488839] LNet: 75343:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 701.681745] Key type .llcrypt unregistered [ 701.684329] Key type ._llcrypt unregistered [ 706.083713] Key type ._llcrypt registered [ 706.085061] Key type .llcrypt registered [ 706.711141] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 22:04:40 (1781229880) [ 706.873332] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 706.882277] alg: No test for adler32 (adler32-zlib) [ 707.705188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 708.315137] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces [ 708.994618] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 22:04:42 (1781229882) [ 709.032092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 709.051196] LNet: 76434:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 709.065844] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 709.096128] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 709.109921] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 709.402987] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 709.413490] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 709.776626] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 709.792045] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp1 [ 709.809890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp1 [ 712.068423] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 22:04:45 (1781229885) [ 712.797215] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_220 Need at least 2 remote nodes found oleg620-server [ 713.408958] Lustre: DEBUG MARKER: == sanity-lnet test 221: Add routes w/DD disabled - check aliveness ========================================================== 22:04:47 (1781229887) [ 714.081749] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_221 Need at least 2 remote nodes found oleg620-server [ 714.739032] Lustre: DEBUG MARKER: == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 22:04:48 (1781229888) [ 715.480956] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_222 Need at least 2 remote nodes found oleg620-server [ 716.185684] Lustre: DEBUG MARKER: == sanity-lnet test 223: Check avoid_asym_router_failure=1 w/DD disabled ========================================================== 22:04:50 (1781229890) [ 716.974747] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_223 Need at least 2 remote nodes found oleg620-server [ 717.650795] Lustre: DEBUG MARKER: == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 22:04:51 (1781229891) [ 718.351358] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_224 Need at least 2 remote nodes found oleg620-server [ 718.972780] Lustre: DEBUG MARKER: == sanity-lnet test 225: Check avoid_asym_router_failure=0 w/DD disabled ========================================================== 22:04:52 (1781229892) [ 719.659041] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_225 Need at least 2 remote nodes found oleg620-server [ 720.317468] Lustre: DEBUG MARKER: == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 22:04:54 (1781229894) [ 720.986483] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_226 Need at least 3 remote nodes found oleg620-server [ 721.599508] Lustre: DEBUG MARKER: == sanity-lnet test 227: Check router peer health w/DD disabled ========================================================== 22:04:55 (1781229895) [ 722.361695] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_227 Need at least 3 remote nodes found oleg620-server [ 723.081199] Lustre: DEBUG MARKER: == sanity-lnet test 228: Routes should stay up when health is decremented ========================================================== 22:04:56 (1781229896) [ 723.845677] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_228 Need at least 3 remote nodes found oleg620-server [ 724.568664] Lustre: DEBUG MARKER: == sanity-lnet test 230: Test setting conns-per-peer ===== 22:04:58 (1781229898) [ 724.609547] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 724.621461] LNet: 80309:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 724.624441] LNetError: 80309:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 724.627599] LNet: Removed LNI 192.168.206.20@tcp [ 725.670997] LNet: Removed LNI 192.168.206.20@tcp1 [ 725.687535] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 725.714492] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 725.726537] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 726.049634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 4 [ 726.083928] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 726.166497] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 726.177540] LNet: 80460:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 726.181699] LNetError: 80460:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 726.187881] LNetError: 80460:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 726.198951] LNet: Removed LNI 192.168.206.20@tcp [ 726.209219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 726.237096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 5 [ 726.249834] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 726.602191] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 726.689680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 727.718427] LNet: Removed LNI 192.168.206.20@tcp [ 727.728634] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 727.760708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 727.774884] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 728.080132] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 6 [ 728.118631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 729.229942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 729.239458] LNet: 80762:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 729.242380] LNet: 80762:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 729.245429] LNetError: 80762:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 729.247988] LNetError: 80762:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 729.257300] LNet: Removed LNI 192.168.206.20@tcp [ 729.265893] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 729.288176] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 7 [ 729.300481] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 729.613049] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 729.685754] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 730.726339] LNet: Removed LNI 192.168.206.20@tcp [ 730.734759] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 730.758481] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 730.770220] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 731.048744] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 8 [ 731.084197] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 731.160738] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 732.198350] LNet: Removed LNI 192.168.206.20@tcp [ 732.206878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 732.231951] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 9 [ 732.245529] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 732.548353] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 732.636801] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 732.647264] LNetError: 81200:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 732.651196] LNetError: 81200:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 732.660319] LNet: Removed LNI 192.168.206.20@tcp [ 732.670670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 732.695595] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 732.707592] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 732.995285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 10 [ 733.030750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 733.119624] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 734.183707] LNet: Removed LNI 192.168.206.20@tcp [ 734.199828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 734.233488] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 11 [ 734.244508] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 734.571555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 735.726278] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 735.735448] LNet: 81501:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 735.738358] LNet: 81501:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 4 previous similar messages [ 736.806398] LNet: Removed LNI 192.168.206.20@tcp [ 736.814870] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 736.837959] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 736.848972] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 737.119359] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 12 [ 737.150375] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 738.295685] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 738.305078] LNetError: 81657:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 738.308778] LNetError: 81657:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 738.316928] LNet: Removed LNI 192.168.206.20@tcp [ 738.326092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 738.348262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 13 [ 738.359936] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 738.676285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 738.765522] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 739.814339] LNet: Removed LNI 192.168.206.20@tcp [ 739.825415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 739.848316] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 739.859519] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 740.157722] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 14 [ 740.189509] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 740.287142] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 741.350573] LNet: Removed LNI 192.168.206.20@tcp [ 741.362080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 741.394631] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 --conns-per-peer 15 [ 741.409503] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 741.732898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 741.837404] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 742.886312] LNet: Removed LNI 192.168.206.20@tcp [ 742.894890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 742.922372] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 742.934604] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 743.219295] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer 128 [ 743.244080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 743.257572] LNet: Removed LNI 192.168.206.20@tcp [ 743.266654] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 743.293377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 743.305725] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 743.623771] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --all --conns-per-peer -1 [ 746.383389] Lustre: DEBUG MARKER: == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 22:05:20 (1781229920) [ 746.443710] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 746.454233] LNet: 82931:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 746.456948] LNet: 82931:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 746.460768] LNetError: 82931:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 746.464053] LNetError: 82931:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 4 previous similar messages [ 746.472625] LNet: Removed LNI 192.168.206.20@tcp [ 746.481089] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 746.506908] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 [ 746.518597] LNet: Added LNI 192.168.206.20@tcp231 [8/256/0/180] [ 747.056409] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 747.081827] LNet: Removed LNI 192.168.206.20@tcp231 [ 747.093107] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 747.125174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 747.138705] LNet: Added LNI 192.168.206.20@tcp231 [8/256/0/180] [ 747.146811] LNet: 83078:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 747.331984] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 747.356733] LNet: Removed LNI 192.168.206.20@tcp231 [ 747.364814] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=-1 [ 747.376495] LNet: Added LNI 192.168.206.20@tcp231 [8/256/0/180] [ 747.877856] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 747.894499] LNet: Removed LNI 192.168.206.20@tcp231 [ 747.906762] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 747.938983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 747.954375] LNet: Added LNI 192.168.206.20@tcp231 [8/256/0/0] [ 747.958423] LNet: 83230:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 748.128204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 748.152415] LNet: Removed LNI 192.168.206.20@tcp231 [ 748.162300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=0 [ 748.178253] LNet: Added LNI 192.168.206.20@tcp231 [8/256/0/0] [ 748.626276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 748.647402] LNet: Removed LNI 192.168.206.20@tcp231 [ 748.655588] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 748.687063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 748.700035] LNet: Added LNI 192.168.206.20@tcp231 [8/256/0/60] [ 748.866716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 748.889825] LNet: Removed LNI 192.168.206.20@tcp231 [ 748.900090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=60 [ 748.913410] LNet: Added LNI 192.168.206.20@tcp231 [8/256/0/60] [ 749.417718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 749.435639] LNet: Removed LNI 192.168.206.20@tcp231 [ 749.446953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 749.483251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-231-expected.yaml [ 749.498676] LNet: Added LNI 192.168.206.20@tcp231 [8/256/0/180] [ 749.503541] LNet: 83533:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 749.507927] LNet: 83533:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 749.683829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp231 --if ens2 [ 749.703484] LNet: Removed LNI 192.168.206.20@tcp231 [ 749.712701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp231 --if ens2 --peer-timeout=180 [ 749.726324] LNet: Added LNI 192.168.206.20@tcp231 [8/256/0/180] [ 752.741477] Lustre: DEBUG MARKER: == sanity-lnet test 232: Test setting ToS value ========== 22:05:26 (1781229926) [ 752.809698] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 752.828642] LNet: Removed LNI 192.168.206.20@tcp231 [ 752.838921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 752.873216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 [ 752.887237] LNet: Added LNI 192.168.206.20@tcp232 [8/256/0/180] [ 753.225151] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 753.247416] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 753.266154] LNet: Removed LNI 192.168.206.20@tcp232 [ 753.277165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos -1 [ 753.295689] LNet: Added LNI 192.168.206.20@tcp232 [8/256/0/180] [ 753.615362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 753.632742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 753.652316] LNet: Removed LNI 192.168.206.20@tcp232 [ 753.660327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 104 [ 753.673185] LNet: Added LNI 192.168.206.20@tcp232 [8/256/0/180] [ 754.006099] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 754.028491] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 754.051156] LNet: Removed LNI 192.168.206.20@tcp232 [ 754.068444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 106 [ 754.082708] LNet: Added LNI 192.168.206.20@tcp232 [8/256/0/180] [ 754.404172] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show --net tcp232 -v 1 [ 754.424046] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp232 --if ens2 [ 754.451135] LNet: Removed LNI 192.168.206.20@tcp232 [ 754.460969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp232 --if ens2 --tos 300 [ 757.025158] Lustre: DEBUG MARKER: == sanity-lnet test 233: Check for successful resends ==== 22:05:30 (1781229930) [ 757.477182] Key type .llcrypt unregistered [ 757.478850] Key type ._llcrypt unregistered [ 759.828254] Key type ._llcrypt registered [ 759.829984] Key type .llcrypt registered [ 759.906208] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 759.912933] alg: No test for adler32 (adler32-zlib) [ 760.731083] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 760.746982] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 764.168828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 767.443216] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 767.460171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 767.471538] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 768.122474] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_interrupt -r 2 [ 768.140377] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e local_interrupt -r 2 [ 768.156112] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_interrupt -r 2 [ 768.176310] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e local_interrupt -r 2 [ 768.195221] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e local_interrupt -r 2 [ 768.215539] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e local_interrupt -r 2 [ 768.238933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e local_interrupt -r 2 [ 768.256681] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e local_interrupt -r 2 [ 768.277406] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.20@tcp 192.168.206.120@tcp [ 768.290560] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 768.294180] LNet: 1 local NIs in recovery (showing 1): 192.168.206.20@tcp [ 768.388687] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 768.402461] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 768.426022] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 768.444824] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 768.459308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 768.477348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 768.495324] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 768.522167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 768.537750] LNet: 86029:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 768.551853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 768.569896] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 768.591878] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 768.611950] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 768.633288] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 768.653694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 768.868054] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_dropped -r 2 [ 768.887510] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e local_dropped -r 2 [ 768.907857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_dropped -r 2 [ 768.927152] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e local_dropped -r 2 [ 768.948181] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e local_dropped -r 2 [ 768.970294] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e local_dropped -r 2 [ 768.998094] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e local_dropped -r 2 [ 769.019169] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e local_dropped -r 2 [ 769.041443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.20@tcp 192.168.206.120@tcp [ 769.053615] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 769.162183] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 769.176846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 769.191956] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 769.208848] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 769.224113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 769.238421] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 769.253960] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 769.268373] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 769.284469] LNet: 86149:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 769.294240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 769.310604] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 769.327761] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 769.346766] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 769.362798] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 769.379647] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 769.615901] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_aborted -r 2 [ 769.636345] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e local_aborted -r 2 [ 769.655562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_aborted -r 2 [ 769.673742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e local_aborted -r 2 [ 769.692356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e local_aborted -r 2 [ 769.713394] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e local_aborted -r 2 [ 769.734199] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e local_aborted -r 2 [ 769.758671] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e local_aborted -r 2 [ 769.775921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.20@tcp 192.168.206.120@tcp [ 769.882910] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 769.903772] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 769.922402] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 769.941954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 769.962212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 769.985116] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 770.011883] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 770.035860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 770.057645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 770.078349] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 770.098229] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 770.116150] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 770.137016] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 770.155283] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 770.408401] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_no_route -r 2 [ 770.427087] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e local_no_route -r 2 [ 770.446240] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_no_route -r 2 [ 770.465830] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e local_no_route -r 2 [ 770.482376] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e local_no_route -r 2 [ 770.500405] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e local_no_route -r 2 [ 770.517682] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e local_no_route -r 2 [ 770.540760] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e local_no_route -r 2 [ 770.567796] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.20@tcp 192.168.206.120@tcp [ 770.580264] LNet: There was an unexpected network error while writing to 192.168.206.120: rc = -22 [ 770.583148] LNet: Skipped 2 previous similar messages [ 770.692362] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 770.711525] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 770.727577] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 770.750612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 770.770763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 770.787259] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 770.801711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 770.819205] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 770.832369] LNet: 86390:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 770.836390] LNet: 86390:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 770.846734] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 770.862299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 770.880148] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 770.896860] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 770.918650] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 770.940189] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 771.174351] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e local_timeout -r 2 [ 771.196265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e local_timeout -r 2 [ 771.216861] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e local_timeout -r 2 [ 771.241215] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e local_timeout -r 2 [ 771.260921] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e local_timeout -r 2 [ 771.279688] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e local_timeout -r 2 [ 771.305194] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e local_timeout -r 2 [ 771.327237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e local_timeout -r 2 [ 771.345306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.20@tcp 192.168.206.120@tcp [ 771.450895] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 771.465127] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 771.481563] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 771.498542] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 771.512545] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 771.529159] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 771.545357] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 771.563171] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 771.579170] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 771.596354] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 771.614320] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 771.633941] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 771.660784] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 771.678101] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 771.929261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp -m GET -e remote_dropped -r 2 [ 771.946716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.120@tcp1 -m GET -e remote_dropped -r 2 [ 771.965261] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp -m GET -e remote_dropped -r 2 [ 771.982096] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp -d 192.168.206.20@tcp1 -m GET -e remote_dropped -r 2 [ 771.999567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp -m GET -e remote_dropped -r 2 [ 772.019809] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s 192.168.206.20@tcp1 -d 192.168.206.120@tcp1 -m GET -e remote_dropped -r 2 [ 772.043837] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp -m GET -e remote_dropped -r 2 [ 772.062088] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s 192.168.206.20@tcp1 -d 192.168.206.20@tcp1 -m GET -e remote_dropped -r 2 [ 772.081703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --source 192.168.206.20@tcp 192.168.206.120@tcp [ 772.093119] LNet: 1 peer NIs in recovery (showing 1): 192.168.206.120@tcp [ 772.197694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 772.215233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 772.232966] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 772.255188] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 772.273527] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 772.294462] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 772.311703] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 772.332961] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 772.349030] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 772.364745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 772.380794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 772.396568] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 772.413972] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 772.431299] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 774.820885] LNet: 86704:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 774.824627] LNetError: 86704:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 774.830717] LNet: Removed LNI 192.168.206.20@tcp [ 774.839358] LNet: Removed LNI 192.168.206.20@tcp1 [ 775.011699] Key type .llcrypt unregistered [ 775.013091] Key type ._llcrypt unregistered [ 781.174174] Key type ._llcrypt registered [ 781.177440] Key type .llcrypt registered [ 781.777278] Lustre: DEBUG MARKER: == sanity-lnet test 234: Verify NI status changes with manual health value transitions ========================================================== 22:05:55 (1781229955) [ 781.940302] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 781.949611] alg: No test for adler32 (adler32-zlib) [ 782.769165] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 782.798444] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 782.811191] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 783.260680] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 0 [ 783.278278] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 0 --nid 192.168.206.20@tcp [ 783.290960] LNetError: 87680:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.206.20@tcp to recovery [ 783.330040] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --nid 192.168.206.20@tcp [ 783.395674] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 786.229327] Lustre: DEBUG MARKER: == sanity-lnet test 235: Verify NI status DOWN when health reaches 0 after failures ========================================================== 22:06:00 (1781229960) [ 786.278418] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 786.295904] LNet: 88264:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 786.302139] LNetError: 88264:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 786.314389] LNet: Removed LNI 192.168.206.20@tcp [ 786.327847] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 786.360321] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 786.374234] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 786.680615] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 786.691386] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 787.109612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 100 --nid 192.168.206.20@tcp [ 787.123639] LNetError: 88544:0:(api-ni.c:4350:lnet_ni_set_healthv()) manually adding local NI 192.168.206.20@tcp to recovery [ 787.360450] LNet: 1 local NIs in recovery (showing 1): 192.168.206.20@tcp [ 787.361117] LNet: There was an unexpected network error while writing to 192.168.206.20: rc = -22 [ 788.217070] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 900 --nid 192.168.206.20@tcp [ 793.077344] Lustre: DEBUG MARKER: == sanity-lnet test 236: Local NI state propagates to routers ========================================================== 22:06:06 (1781229966) [ 793.938212] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_236 Need at least 2 remote nodes found oleg620-server [ 794.676790] Lustre: DEBUG MARKER: == sanity-lnet test 237: Check MR forwarding feature ===== 22:06:08 (1781229968) [ 795.493157] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_237 Need at least 2 remote nodes found oleg620-server [ 796.273753] Lustre: DEBUG MARKER: == sanity-lnet test 238: Check MR forwarding feature skipped for non-MR source ========================================================== 22:06:10 (1781229970) [ 797.069963] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_238 Need at least 2 remote nodes found oleg620-server [ 797.837484] Lustre: DEBUG MARKER: == sanity-lnet test 241: Check setting LND timeout value via lnetctl updates tunables ========================================================== 22:06:11 (1781229971) [ 797.914715] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 797.926758] LNet: 90172:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 797.930831] LNetError: 90172:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 797.937237] LNet: Removed LNI 192.168.206.20@tcp [ 797.951251] LNet: Removed LNI 192.168.206.20@tcp1 [ 797.961048] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 797.989601] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 798.001516] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 798.344593] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set -h [ 798.369484] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 51 [ 798.390250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 798.422533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 0 [ 798.449602] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 798.473219] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --net tcp --lnd-timeout 50 [ 801.178239] Lustre: DEBUG MARKER: == sanity-lnet test 245: Check read-only params do not appear in export --backup ========================================================== 22:06:15 (1781229975) [ 801.216231] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 801.226930] LNet: 90892:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 801.230132] LNetError: 90892:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 801.234384] LNetError: 90892:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 801.248733] LNet: Removed LNI 192.168.206.20@tcp [ 801.263407] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 801.290751] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 801.302164] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 804.253901] Lustre: DEBUG MARKER: == sanity-lnet test 250: test that linux routes are added ========================================================== 22:06:18 (1781229978) [ 804.301180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 804.313243] LNet: 91596:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 804.317812] LNetError: 91596:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 804.329286] LNet: Removed LNI 192.168.206.20@tcp [ 804.342691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 804.367826] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 804.382976] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 807.220977] Lustre: DEBUG MARKER: == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 22:06:21 (1781229981) [ 807.849377] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_251 Need kfi NETTYPE [ 808.561610] Lustre: DEBUG MARKER: == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 22:06:22 (1781229982) [ 808.738256] LNet: 92634:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 808.743871] LNetError: 92634:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 808.753845] LNet: Removed LNI 192.168.206.20@tcp [ 809.018221] Key type .llcrypt unregistered [ 809.019879] Key type ._llcrypt unregistered [ 811.343242] Key type ._llcrypt registered [ 811.344944] Key type .llcrypt registered [ 811.437686] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 811.445280] alg: No test for adler32 (adler32-zlib) [ 812.268460] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 812.282971] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 815.554560] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 817.264046] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping --timeout 15 192.168.206.120@tcp [ 817.388939] LNet: 93049:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 817.392666] LNetError: 93049:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 817.402971] LNet: Removed LNI 192.168.206.20@tcp [ 817.569444] Key type .llcrypt unregistered [ 817.571128] Key type ._llcrypt unregistered [ 821.805452] Key type ._llcrypt registered [ 821.807152] Key type .llcrypt registered [ 822.353570] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253 [ 822.981843] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254 [ 823.621976] Lustre: DEBUG MARKER: == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 22:06:37 (1781229997) [ 824.256068] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_255 kernel ip2nets deprecated in 2.17.53 [ 824.937393] Lustre: DEBUG MARKER: == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 22:06:38 (1781229998) [ 825.744929] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_256 Need at least 2 remote nodes found oleg620-server [ 826.449809] Lustre: DEBUG MARKER: == sanity-lnet test 257: Test graceful router shutdown/startup ========================================================== 22:06:40 (1781230000) [ 827.208427] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_257 Need at least 3 remote nodes found oleg620-server [ 827.901803] Lustre: DEBUG MARKER: == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 22:06:41 (1781230001) [ 828.380123] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 828.399098] alg: No test for adler32 (adler32-zlib) [ 829.300632] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 829.408238] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 831.000254] Key type lgssc registered [ 831.527451] Lustre: Echo OBD driver; http://www.lustre.org/ [ 840.861357] Key type lgssc unregistered [ 841.003579] LNet: 96217:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 841.007503] LNetError: 96217:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 841.018449] LNet: Removed LNI 192.168.206.20@tcp [ 841.338628] Key type .llcrypt unregistered [ 841.340882] Key type ._llcrypt unregistered [ 847.588751] Key type ._llcrypt registered [ 847.591841] Key type .llcrypt registered [ 848.206119] Lustre: DEBUG MARKER: == sanity-lnet test 265: Import of invalid CPT should fail ========================================================== 22:07:01 (1781230021) [ 848.429294] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 848.440877] alg: No test for adler32 (adler32-zlib) [ 849.283352] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 849.322843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 849.347576] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 849.370300] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 849.401716] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-265.yaml [ 853.038349] Lustre: DEBUG MARKER: == sanity-lnet test 266: Validate CPT parsing in network sequence ========================================================== 22:07:06 (1781230026) [ 853.140060] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 853.152457] LNet: 97630:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 853.165739] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 853.193281] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 853.205935] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 853.783953] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 853.795669] LNet: 97774:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 853.799275] LNetError: 97774:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 853.811470] LNet: Removed LNI 192.168.206.20@tcp [ 853.825298] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 853.859034] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 853.875772] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 854.137580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 854.160086] LNet: Removed LNI 192.168.206.20@tcp [ 854.173488] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 854.197690] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-266.yaml [ 854.213620] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 857.377454] Lustre: DEBUG MARKER: == sanity-lnet test 270: Test NID metadata =============== 22:07:11 (1781230031) [ 857.440151] LNet: 98388:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 857.443829] LNet: 98388:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 857.448353] LNetError: 98388:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 857.454358] LNetError: 98388:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 857.466691] LNet: Removed LNI 192.168.206.20@tcp [ 857.752733] Key type .llcrypt unregistered [ 857.754600] Key type ._llcrypt unregistered [ 860.369181] Key type ._llcrypt registered [ 860.371089] Key type .llcrypt registered [ 860.463250] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 860.470444] alg: No test for adler32 (adler32-zlib) [ 861.289640] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 861.326467] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 861.328856] LNet: *** cfs_fail_loc=e003, val=100*** [ 861.637957] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 861.649288] Ping reply received with mapping: [ 861.651021] found metadata reply NI[192.168.206.20@tcp] [ 861.652603] 14 ce a8 c0 00 00 02 00 5a 5a 5a 5a 5a 5a 5a 5a [ 861.655260] 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a [ 861.657203] 5a 5a 5a 5a 5a 5a 5a 5a [ 861.707576] LNet: 98789:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 861.712576] LNetError: 98789:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 861.722494] LNet: Removed LNI 192.168.206.20@tcp [ 862.004804] Key type .llcrypt unregistered [ 862.007346] Key type ._llcrypt unregistered [ 866.786272] Key type ._llcrypt registered [ 866.788372] Key type .llcrypt registered [ 867.420325] Lustre: DEBUG MARKER: == sanity-lnet test 280: Don't panic when request_module fails ========================================================== 22:07:21 (1781230041) [ 867.614330] Key type .llcrypt unregistered [ 867.615478] Key type ._llcrypt unregistered [ 870.018752] Key type ._llcrypt registered [ 870.020663] Key type .llcrypt registered [ 870.108797] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 870.116465] alg: No test for adler32 (adler32-zlib) [ 871.021146] LNetError: 99833:0:(api-ni.c:2729:lnet_load_lnd()) Can't load LND tcp, module ksocklnd, rc=256 [ 871.180711] Key type .llcrypt unregistered [ 871.182148] Key type ._llcrypt unregistered [ 873.307967] Key type ._llcrypt registered [ 873.309675] Key type .llcrypt registered [ 873.943617] Lustre: DEBUG MARKER: == sanity-lnet test 290: Check that lnet acceptor is using explicit address binding ========================================================== 22:07:27 (1781230047) [ 874.162521] Key type .llcrypt unregistered [ 874.164350] Key type ._llcrypt unregistered [ 876.417785] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 876.419879] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 878.645046] Key type ._llcrypt registered [ 878.646384] Key type .llcrypt registered [ 878.729840] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 878.737221] alg: No test for adler32 (adler32-zlib) [ 879.551178] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 879.576556] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 883.866698] Lustre: DEBUG MARKER: == sanity-lnet test 291: load lnet using custom accept port for bulk tcp connections ========================================================== 22:07:37 (1781230057) [ 883.939253] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 883.950565] LNet: 101611:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 883.954947] LNetError: 101611:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 883.963524] LNet: Removed LNI 10.1.2.3@tcp [ 883.975622] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 884.284596] Key type .llcrypt unregistered [ 884.285770] Key type ._llcrypt unregistered [ 886.660773] Key type ._llcrypt registered [ 886.662712] Key type .llcrypt registered [ 886.747389] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 886.757949] alg: No test for adler32 (adler32-zlib) [ 887.576532] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 887.601498] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 887.667117] LNet: 101899:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 887.670579] LNetError: 101899:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 887.680441] LNet: Removed LNI 192.168.206.20@tcp [ 887.973563] Key type .llcrypt unregistered [ 887.975286] Key type ._llcrypt unregistered [ 892.250439] Key type ._llcrypt registered [ 892.251886] Key type .llcrypt registered [ 892.780224] Lustre: DEBUG MARKER: == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 22:07:46 (1781230066) [ 893.043443] Key type .llcrypt unregistered [ 893.045083] Key type ._llcrypt unregistered [ 895.343680] Key type ._llcrypt registered [ 895.345552] Key type .llcrypt registered [ 895.422149] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 895.429684] alg: No test for adler32 (adler32-zlib) [ 898.768206] Lustre: DEBUG MARKER: == sanity-lnet test 301: Check for dynamic adds of same/wrong interface (memory leak) ========================================================== 22:07:52 (1781230072) [ 898.811750] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 898.835427] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 898.870389] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 898.883750] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 899.180656] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 899.197272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net del --net tcp --if ens2 [ 899.214701] LNet: Removed LNI 192.168.206.20@tcp [ 899.313420] LNet: 103639:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 899.498764] Key type .llcrypt unregistered [ 899.500727] Key type ._llcrypt unregistered [ 903.792601] Key type ._llcrypt registered [ 903.794339] Key type .llcrypt registered [ 904.390878] Lustre: DEBUG MARKER: == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 22:07:58 (1781230078) [ 904.560938] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 904.567789] alg: No test for adler32 (adler32-zlib) [ 905.384713] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 905.410804] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 905.425871] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 905.779424] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 905.803684] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl debug peer --nid 192.168.206.20@tcp [ 905.814733] LNet: Conn 0 [type, closing, crefcnt, srefcnt]: 2, 0, 1, 1 [ 905.817119] LNet: Conn 0 rx [scheduled, ready, state]: 0, 0, 1 [ 905.819377] LNet: Conn 0 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 905, 0, 0 [ 905.822590] LNet: Conn 0 sched [nconns, cpt]: 4, 0 [ 905.824538] LNet: Conn 1 [type, closing, crefcnt, srefcnt]: 3, 0, 1, 1 [ 905.827284] LNet: Conn 1 rx [scheduled, ready, state]: 0, 0, 1 [ 905.830209] LNet: Conn 1 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 905, 0, 0 [ 905.833646] LNet: Conn 1 sched [nconns, cpt]: 4, 0 [ 905.835507] LNet: Conn 2 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 905.838101] LNet: Conn 2 rx [scheduled, ready, state]: 0, 0, 1 [ 905.839982] LNet: Conn 2 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 905, 0, 955 [ 905.842689] LNet: Conn 2 sched [nconns, cpt]: 4, 0 [ 905.844316] LNet: Conn 3 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1 [ 905.846458] LNet: Conn 3 rx [scheduled, ready, state]: 0, 0, 1 [ 905.850143] LNet: Conn 3 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 905, 0, 955 [ 905.853615] LNet: Conn 3 sched [nconns, cpt]: 4, 0 [ 908.465908] Lustre: DEBUG MARKER: == sanity-lnet test 303: Check peer NI health after link down ========================================================== 22:08:02 (1781230082) [ 908.629468] LNet: 105179:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 908.633456] LNetError: 105179:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 908.640713] LNet: Removed LNI 192.168.206.20@tcp [ 908.887481] Key type .llcrypt unregistered [ 908.889159] Key type ._llcrypt unregistered [ 911.098296] Key type ._llcrypt registered [ 911.100030] Key type .llcrypt registered [ 911.186357] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 911.197473] alg: No test for adler32 (adler32-zlib) [ 912.037694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 912.053530] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 915.370522] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 918.496250] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 918.522306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 918.535730] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 918.969717] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 919.017630] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 919.021249] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 919.023337] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 921.195513] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp99 --if test1pg [ 921.208849] LNet: Added LNI 10.1.2.3@tcp99 [8/256/0/180] [ 925.624096] Lustre: DEBUG MARKER: == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 22:08:19 (1781230099) [ 925.674276] LNet: 106675:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 925.678172] LNetError: 106675:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 925.682444] LNet: Removed LNI 192.168.206.20@tcp [ 926.688310] LNet: Removed LNI 192.168.206.20@tcp1 [ 926.690100] LNetError: 106675:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface test1pg not found [ 926.693125] LNetError: 106675:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 1 previous similar message [ 926.702747] LNet: Removed LNI 10.1.2.3@tcp99 [ 926.981704] Key type .llcrypt unregistered [ 926.983342] Key type ._llcrypt unregistered [ 929.186608] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 929.188896] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 931.380662] Key type ._llcrypt registered [ 931.382306] Key type .llcrypt registered [ 931.453572] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 931.460875] alg: No test for adler32 (adler32-zlib) [ 932.271262] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 932.295939] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 932.309512] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 932.622902] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 932.633388] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 933.502691] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim_nid 10.1.2.3@tcp --lock_prim [ 933.523350] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 933.543980] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --nid 10.1.2.3@tcp --state 1048577 [ 933.555569] LNetError: 107424:0:(peer.c:2175:lnet_del_peer_ni()) peer 10.1.2.3@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs [ 933.571818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 933.593237] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 933.755843] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer show -v 4 --nid 10.1.2.3@tcp [ 933.789453] LNet: 107444:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 933.793536] LNetError: 107444:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 933.802092] LNet: Removed LNI 192.168.206.20@tcp [ 933.810313] LNet: Removed LNI 10.1.2.3@tcp [ 934.094741] Key type .llcrypt unregistered [ 934.096612] Key type ._llcrypt unregistered [ 938.321613] Key type ._llcrypt registered [ 938.323338] Key type .llcrypt registered [ 938.865945] Lustre: DEBUG MARKER: == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 22:08:32 (1781230112) [ 938.992480] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 938.998753] alg: No test for adler32 (adler32-zlib) [ 939.809748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 939.836466] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 939.849623] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 940.151835] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping 192.168.206.20@tcp [ 940.183670] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl ping oleg620-client@tcp [ 942.574735] Lustre: DEBUG MARKER: == sanity-lnet test 310: Set timeout and verify ========== 22:08:36 (1781230116) [ 942.615683] LNet: 108967:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 942.621658] LNetError: 108967:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 942.635515] LNet: Removed LNI 192.168.206.20@tcp [ 942.925534] Key type .llcrypt unregistered [ 942.927268] Key type ._llcrypt unregistered [ 945.112959] Key type ._llcrypt registered [ 945.114676] Key type .llcrypt registered [ 945.180319] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 945.186919] alg: No test for adler32 (adler32-zlib) [ 945.992806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure --all [ 946.006212] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 946.022494] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net show -v [ 948.769705] Lustre: DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 22:08:42 (1781230122) [ 949.412424] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type [ 950.119283] Lustre: DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 22:08:43 (1781230123) [ 950.807838] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type [ 951.651962] Lustre: DEBUG MARKER: == sanity-lnet test 350: Check import --exec of ping/discover ========================================================== 22:08:45 (1781230125) [ 951.706443] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 951.728298] LNet: 110387:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 951.736606] LNetError: 110387:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 951.746273] LNet: Removed LNI 192.168.206.20@tcp [ 951.756187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 951.781984] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 951.796401] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 952.180763] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 952.205427] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import -e /tmp/sanity-lnet-350.yaml [ 955.579635] Lustre: DEBUG MARKER: == sanity-lnet test 400: Check for udsp add/delete net rule without net num ========================================================== 22:08:49 (1781230129) [ 955.618081] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 955.628481] LNet: 111094:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 955.634899] LNetError: 111094:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 955.653589] LNet: Removed LNI 192.168.206.20@tcp [ 955.663580] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 955.690841] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --priority 0 [ 955.705621] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 956.050434] Key type .llcrypt unregistered [ 956.053109] Key type ._llcrypt unregistered [ 960.931516] Key type ._llcrypt registered [ 960.933109] Key type .llcrypt registered [ 961.579474] Lustre: DEBUG MARKER: == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 22:08:55 (1781230135) [ 961.855673] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 961.869572] alg: No test for adler32 (adler32-zlib) [ 962.728080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 962.757933] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 962.774389] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 963.175924] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 963.214857] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 966.402313] Lustre: DEBUG MARKER: == sanity-lnet test 402: Destination net rule should not panic ========================================================== 22:09:00 (1781230140) [ 966.452745] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 966.468668] LNet: 112649:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 966.474122] LNetError: 112649:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 966.483188] LNet: Removed LNI 192.168.206.20@tcp [ 966.493735] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 966.530805] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst kfi --priority 0 [ 966.550285] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer add --prim 402@kfi [ 969.196593] Lustre: DEBUG MARKER: == sanity-lnet test 403: Add and delete udsp net and nid rules ========================================================== 22:09:03 (1781230143) [ 969.272571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 969.281288] LNet: 113236:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 969.292969] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 969.320917] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 969.467242] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 969.484612] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 969.509212] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src 1.1.1.1@tcp --prio 1 [ 969.661415] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 969.679552] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 969.713940] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 969.731708] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 969.772794] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 969.933910] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 969.951241] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 969.980378] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst 1.1.1.1@tcp --prio 1 [ 970.132817] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp del --idx 0 [ 970.148793] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp show [ 972.946086] Lustre: DEBUG MARKER: == sanity-lnet test 404: Check udsp src/dst net rule ===== 22:09:06 (1781230146) [ 972.993238] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 973.005958] LNet: 113863:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 973.012073] LNet: 113863:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 973.027388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 973.059922] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 973.073554] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 973.428937] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 973.442199] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 973.865635] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 973.891850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 973.980306] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 973.992570] LNetError: 114148:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 975.008133] LNet: Removed LNI 192.168.206.20@tcp [ 975.009742] LNetError: 114148:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 975.018889] LNet: Removed LNI 192.168.206.20@tcp2 [ 975.030264] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 975.062693] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 975.065754] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 975.068796] LNet: 114154:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 975.321483] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 975.427403] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 975.440115] LNet: 114191:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 975.443482] LNet: 114191:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 976.480316] LNet: Removed LNI 192.168.206.20@tcp [ 976.490637] LNet: Removed LNI 192.168.206.20@tcp2 [ 976.513594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 976.549936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 976.563231] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 976.951313] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 976.963789] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 977.361379] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 977.381846] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 977.413915] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 977.536571] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 977.549693] LNetError: 114475:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 977.553273] LNetError: 114475:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 2 previous similar messages [ 978.592117] LNet: Removed LNI 192.168.206.20@tcp [ 979.625548] LNet: Removed LNI 192.168.206.20@tcp2 [ 979.638487] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 979.672121] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 979.678862] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 979.684467] LNet: 114482:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 979.951646] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 980.065195] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 980.076742] LNet: 114519:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 980.080660] LNet: 114519:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 1 previous similar message [ 980.083713] LNet: Removed LNI 192.168.206.20@tcp [ 981.094596] LNet: Removed LNI 192.168.206.20@tcp2 [ 981.109903] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 981.142890] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 981.157642] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 981.512540] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 981.530834] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 981.918291] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 981.950303] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 982.039174] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 982.050554] LNetError: 114798:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 982.054872] LNetError: 114798:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 3 previous similar messages [ 982.058972] LNet: Removed LNI 192.168.206.20@tcp [ 982.065585] LNet: Removed LNI 192.168.206.20@tcp2 [ 982.075689] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 982.104026] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 982.107147] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 982.109907] LNet: 114804:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 982.323748] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 982.428080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 983.456986] LNet: Removed LNI 192.168.206.20@tcp [ 983.465431] LNet: Removed LNI 192.168.206.20@tcp2 [ 983.477393] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 983.509618] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 983.521836] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 983.863293] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 983.882093] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 984.266876] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 984.287555] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 984.312764] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 984.408652] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 985.440143] LNet: Removed LNI 192.168.206.20@tcp [ 986.470434] LNet: Removed LNI 192.168.206.20@tcp2 [ 986.480113] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 986.512112] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 986.515810] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 986.519037] LNet: 115131:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 986.713688] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 986.797954] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 986.809991] LNet: Removed LNI 192.168.206.20@tcp [ 987.879531] LNet: Removed LNI 192.168.206.20@tcp2 [ 987.889653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 987.914653] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 987.927399] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 988.273236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 988.281796] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 988.670501] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp --prio 1 [ 988.687120] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp --prio 1 [ 988.714368] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 988.884251] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 988.895805] LNet: 115465:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 988.900415] LNet: 115465:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 4 previous similar messages [ 989.920196] LNet: Removed LNI 192.168.206.20@tcp [ 989.928452] LNet: Removed LNI 192.168.206.20@tcp2 [ 989.937655] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 989.965774] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 989.969541] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 990.191326] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 990.349193] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 990.362507] LNetError: 115523:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 990.366561] LNetError: 115523:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 9 previous similar messages [ 990.371119] LNet: Removed LNI 192.168.206.20@tcp [ 990.378516] LNet: Removed LNI 192.168.206.20@tcp2 [ 990.390688] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 990.415567] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 990.426475] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 990.745400] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp2 --if ens2 [ 990.757424] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 991.150707] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 991.173850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --src tcp2 --prio 1 [ 991.189092] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl udsp add --dst tcp2 --prio 1 [ 991.221388] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 991.387974] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 992.416141] LNet: Removed LNI 192.168.206.20@tcp [ 993.446529] LNet: Removed LNI 192.168.206.20@tcp2 [ 993.458156] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 993.493172] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 993.499303] LNet: Added LNI 192.168.206.20@tcp2 [8/256/0/180] [ 993.502740] LNet: 115830:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 993.505527] LNet: 115830:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 993.731420] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 996.898517] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_410 skipping ALWAYS excluded test 410 [ 997.666261] Lustre: DEBUG MARKER: == sanity-lnet test 450: Check import of multiple interfaces ========================================================== 22:09:31 (1781230171) [ 997.750104] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 997.765693] LNet: Removed LNI 192.168.206.20@tcp [ 998.825241] LNet: Removed LNI 192.168.206.20@tcp2 [ 998.843550] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 998.998410] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 999.019482] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 999.371495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 999.384462] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 999.843871] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 999.859927] LNet: Removed LNI 192.168.206.20@tcp [ 999.879524] LNet: Removed LNI 10.1.2.3@tcp [ 999.892040] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 999.924348] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import /tmp/sanity-lnet-450.yaml [ 999.942731] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 999.946522] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1000.140594] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import --del /tmp/sanity-lnet-450.yaml [ 1000.157861] LNet: Removed LNI 192.168.206.20@tcp [ 1000.172428] LNet: Removed LNI 10.1.2.3@tcp [ 1003.519398] Lustre: DEBUG MARKER: == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 22:09:37 (1781230177) [ 1003.557533] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1003.599173] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1003.674563] IPv6: ADDRCONF(NETDEV_UP): test1pl: link is not ready [ 1003.716492] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1003.720825] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1003.726506] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready [ 1005.931741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1005.945403] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 1006.277064] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if test1pg [ 1006.292083] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180] [ 1007.277167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.20@tcp [ 1007.296936] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault delay add -s *@tcp -d *@tcp -r 1 -l 1 -m PUT [ 1007.323343] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 192.168.206.20@tcp -m PUT -e local_timeout -r 1 [ 1007.353439] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d 10.1.2.3@tcp -m PUT -e local_timeout -r 1 [ 1007.387880] IPv6: ADDRCONF(NETDEV_UP): test1pg: link is not ready [ 1007.393975] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready [ 1007.521572] LNetError: 117529:0:(lib-move.c:4710:lnet_parse()) 192.168.206.20@tcp, src 192.168.206.20@tcp: Dropping PUT (error -108 looking up sender) [ 1008.545104] LNet: 117988:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1008.553372] LNet: 117988:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Skipped 5 previous similar messages [ 1008.561834] LNetError: 117988:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1008.569259] LNetError: 117988:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Skipped 7 previous similar messages [ 1008.574499] LNet: Removed LNI 192.168.206.20@tcp [ 1008.585246] LNet: Removed LNI 10.1.2.3@tcp [ 1008.797876] Key type .llcrypt unregistered [ 1008.800273] Key type ._llcrypt unregistered [ 1013.439894] Key type ._llcrypt registered [ 1013.441810] Key type .llcrypt registered [ 1014.093208] Lustre: DEBUG MARKER: == sanity-lnet test 501: Verify lnetctl peer set --health (SR) ========================================================== 22:09:47 (1781230187) [ 1014.240932] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1014.251405] alg: No test for adler32 (adler32-zlib) [ 1015.078558] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1015.244984] LNet: 118839:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1015.502611] Key type .llcrypt unregistered [ 1015.504213] Key type ._llcrypt unregistered [ 1017.958615] Key type ._llcrypt registered [ 1017.961685] Key type .llcrypt registered [ 1018.080988] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1018.088414] alg: No test for adler32 (adler32-zlib) [ 1018.903514] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1018.918260] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 1022.281742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 1022.386329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl fault drop add -s *@tcp -d *@tcp -r 1 [ 1022.653942] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.206.120@tcp [ 1022.778562] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1022.797786] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1022.814180] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1022.828905] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1022.849649] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1022.867399] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1022.889124] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1022.911454] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1022.925337] LNet: 119287:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1022.940889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1022.962633] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1022.977395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1022.994967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1023.009822] LNet: 1 peer NIs in recovery (showing 1): 192.168.206.120@tcp [ 1023.014044] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1023.031807] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1023.338644] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 1023.463451] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1023.478967] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1023.497185] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1023.515204] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1023.530909] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1023.549879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1023.567977] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1023.591727] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1023.605885] LNet: 119404:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1023.617695] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1023.635214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1023.650276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1023.673945] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1023.692167] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1023.714549] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1024.053450] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.206.120@tcp [ 1024.182524] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1024.202898] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1024.221246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1024.238479] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1024.258963] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1024.284790] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1024.306526] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1024.323236] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1024.345779] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1024.375279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1024.402632] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1024.424505] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1024.442210] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1024.467657] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1024.791736] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 1024.901333] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1024.919485] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1024.939743] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1024.957329] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1024.975528] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1024.993828] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1025.016437] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1025.037233] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1025.050964] LNet: 119639:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1025.054070] LNet: 119639:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 1025.063179] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1025.081931] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1025.099825] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1025.122276] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1025.145143] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1025.166854] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1025.317824] LNet: 119685:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1025.321863] LNetError: 119685:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1025.333439] LNet: Removed LNI 192.168.206.20@tcp [ 1025.515507] Key type .llcrypt unregistered [ 1025.516862] Key type ._llcrypt unregistered [ 1032.894778] Key type ._llcrypt registered [ 1032.896299] Key type .llcrypt registered [ 1033.566351] Lustre: DEBUG MARKER: == sanity-lnet test 502: Verify lnetctl peer set --health (MR) ========================================================== 22:10:07 (1781230207) [ 1033.742878] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1033.750558] alg: No test for adler32 (adler32-zlib) [ 1034.569495] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1034.748330] LNet: 120570:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1035.067476] Key type .llcrypt unregistered [ 1035.069083] Key type ._llcrypt unregistered [ 1037.675191] Key type ._llcrypt registered [ 1037.676913] Key type .llcrypt registered [ 1037.789111] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1037.802279] alg: No test for adler32 (adler32-zlib) [ 1038.636770] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure -a [ 1038.650733] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 1042.277850] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl discover 192.168.206.120@tcp [ 1046.156701] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1046.178885] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp1 --if ens2 [ 1046.193298] LNet: Added LNI 192.168.206.20@tcp1 [8/256/0/180] [ 1046.645982] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net_drop add -s *@tcp -d *@tcp -r 1 [ 1046.900531] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --nid 192.168.206.120@tcp [ 1047.022829] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1047.041063] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1047.057247] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1047.072255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1047.088374] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1047.106741] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1047.123482] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1047.144225] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1047.153401] LNet: 121220:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1047.164289] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1047.183395] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1047.200780] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1047.222327] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1047.243355] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1047.262983] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1047.544853] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 0 --all [ 1047.643277] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1047.660279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1047.678214] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1047.696149] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1047.716268] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1047.731257] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1047.749308] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1047.767587] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1047.781331] LNet: 121337:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1047.793080] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1047.812104] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1047.842255] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1047.859806] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1047.879617] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1047.895486] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1048.186711] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --nid 192.168.206.120@tcp [ 1048.297639] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1048.317158] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1048.336879] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1048.353932] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1048.378265] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1048.399095] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1048.417391] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1048.437818] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1048.461246] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1048.485282] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1048.501307] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1048.516243] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1048.532090] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1048.550280] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1048.836086] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 500 --all [ 1048.928098] LNet: 2 peer NIs in recovery (showing 2): 192.168.206.120@tcp, 192.168.206.120@tcp1 [ 1048.960645] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set numa_range 0 [ 1048.986356] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_interfaces 200 [ 1049.004718] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set discovery 1 [ 1049.021551] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set drop_asym_route 0 [ 1049.039279] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set retry_count 2 [ 1049.055889] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set transaction_timeout 150 [ 1049.078666] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set health_sensitivity 100 [ 1049.096522] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_interval 1 [ 1049.106697] LNet: 121571:0:(api-ni.c:294:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated [ 1049.111699] LNet: 121571:0:(api-ni.c:294:recovery_interval_set()) Skipped 1 previous similar message [ 1049.123187] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set router_sensitivity 100 [ 1049.142032] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set response_tracking 3 [ 1049.161742] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set recovery_limit 0 [ 1049.178767] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl set max_recovery_ping_interval 900 [ 1049.200572] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl peer set --health 1000 --all [ 1049.218627] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net set --health 1000 --all [ 1051.705491] LNet: 121651:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1051.710641] LNetError: 121651:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1051.714549] LNet: Removed LNI 192.168.206.20@tcp [ 1051.722174] LNet: Removed LNI 192.168.206.20@tcp1 [ 1051.924883] Key type .llcrypt unregistered [ 1051.926797] Key type ._llcrypt unregistered [ 1059.198501] Key type ._llcrypt registered [ 1059.200011] Key type .llcrypt registered [ 1059.900830] Lustre: DEBUG MARKER: == sanity-lnet test 600: lnet_legacy2yaml: basic ip2nets conversion ========================================================== 22:10:33 (1781230233) [ 1063.015132] Lustre: DEBUG MARKER: == sanity-lnet test 601: lnet_legacy2yaml: basic routes conversion ========================================================== 22:10:36 (1781230236) [ 1066.221214] Lustre: DEBUG MARKER: == sanity-lnet test 602: lnet_legacy2yaml: routes with priority ========================================================== 22:10:40 (1781230240) [ 1069.647667] Lustre: DEBUG MARKER: == sanity-lnet test 603: lnet_legacy2yaml: default hop flag ========================================================== 22:10:43 (1781230243) [ 1073.205178] Lustre: DEBUG MARKER: == sanity-lnet test 604: lnet_legacy2yaml: default hop with explicit value ========================================================== 22:10:46 (1781230246) [ 1076.803889] Lustre: DEBUG MARKER: == sanity-lnet test 605: lnet_legacy2yaml: quote leading star in IP pattern ========================================================== 22:10:50 (1781230250) [ 1080.614165] Lustre: DEBUG MARKER: == sanity-lnet test 606: lnet_legacy2yaml: file inputs === 22:10:54 (1781230254) [ 1084.473421] Lustre: DEBUG MARKER: == sanity-lnet test 607: lnet_legacy2yaml: options line with combined ip2nets and routes ========================================================== 22:10:58 (1781230258) [ 1088.386398] Lustre: DEBUG MARKER: == sanity-lnet test 608: lnet_legacy2yaml: error on invalid rule (missing IP pattern) ========================================================== 22:11:01 (1781230261) [ 1091.584474] Lustre: DEBUG MARKER: == sanity-lnet test 609: lnet_legacy2yaml: error on no input ========================================================== 22:11:05 (1781230265) [ 1094.678358] Lustre: DEBUG MARKER: == sanity-lnet test 610: lnet_legacy2yaml: multiple files and inline, check order ========================================================== 22:11:08 (1781230268) [ 1098.366696] Lustre: DEBUG MARKER: == sanity-lnet test 611: lnet_legacy2yaml: invalid hopcount ========================================================== 22:11:12 (1781230272) [ 1101.627882] Lustre: DEBUG MARKER: == sanity-lnet test 612: lnet_legacy2yaml: malformed IP pattern ========================================================== 22:11:15 (1781230275) [ 1104.925905] Lustre: DEBUG MARKER: == sanity-lnet test 613: lnet_legacy2yaml: multiple IP patterns in single rule ========================================================== 22:11:18 (1781230278) [ 1108.354804] Lustre: DEBUG MARKER: == sanity-lnet test 614: lnet_legacy2yaml: interfaces enumeration with multiple IP ranges ========================================================== 22:11:22 (1781230282) [ 1111.811856] Lustre: DEBUG MARKER: == sanity-lnet test 615: lnet_legacy2yaml: modprobe configuration file input ========================================================== 22:11:25 (1781230285) [ 1115.448509] Lustre: DEBUG MARKER: == sanity-lnet test 616: lnet_legacy2yaml: combining modprobe file with explicit ip2nets file ========================================================== 22:11:29 (1781230289) [ 1118.572789] Lustre: DEBUG MARKER: == sanity-lnet test 617: lnet_legacy2yaml: round-trip tunables from LNet to modprobe to YAML ========================================================== 22:11:32 (1781230292) [ 1118.727683] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1118.735553] alg: No test for adler32 (adler32-zlib) [ 1119.555553] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1119.616138] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl net add --net tcp --if ens2 [ 1119.629105] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 1120.163089] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet unconfigure [ 1120.176175] LNet: 132347:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1120.180591] LNetError: 132347:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1120.199877] LNet: Removed LNI 192.168.206.20@tcp [ 1120.211272] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl lnet configure [ 1120.236694] Lustre: DEBUG MARKER: /home/green/git/lustre-release/lustre/../lnet/utils/lnetctl import [ 1120.250355] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 1123.259889] Lustre: DEBUG MARKER: == sanity-lnet test 618: lnet_legacy2yaml: alternative route format ========================================================== 22:11:37 (1781230297) [ 1126.382144] Lustre: DEBUG MARKER: == sanity-lnet test 619: lnet_legacy2yaml: multiple alternative format routes from file ========================================================== 22:11:40 (1781230300) [ 1129.532588] Lustre: DEBUG MARKER: == sanity-lnet test 620: lnet_legacy2yaml: mixed legacy and alternative route formats ========================================================== 22:11:43 (1781230303) [ 1132.457674] Lustre: DEBUG MARKER: == sanity-lnet test 621: lnet_legacy2yaml: alternative format with invalid hopcount ========================================================== 22:11:46 (1781230306) [ 1135.035033] Lustre: DEBUG MARKER: == sanity-lnet test 622: lnet_legacy2yaml: alternative format with default-hop flag ========================================================== 22:11:48 (1781230308) [ 1137.848367] Lustre: DEBUG MARKER: == sanity-lnet test 623: lnet_legacy2yaml: basic networks conversion ========================================================== 22:11:51 (1781230311) [ 1140.835181] Lustre: DEBUG MARKER: == sanity-lnet test 624: lnet_legacy2yaml: networks with interfaces ========================================================== 22:11:54 (1781230314) [ 1143.902203] Lustre: DEBUG MARKER: == sanity-lnet test 625: lnet_legacy2yaml: multiple networks ========================================================== 22:11:57 (1781230317) [ 1147.153978] Lustre: DEBUG MARKER: == sanity-lnet test 626: lnet_legacy2yaml: mixed networks, ip2nets, and routes ========================================================== 22:12:00 (1781230320) [ 1150.235808] Lustre: DEBUG MARKER: == sanity-lnet test 627: lnet_legacy2yaml: invalid networks format ========================================================== 22:12:04 (1781230324) [ 1153.307753] Lustre: DEBUG MARKER: == sanity-lnet test 628: lnet_legacy2yaml: missing delimiter error ========================================================== 22:12:07 (1781230327) [ 1156.298481] Lustre: DEBUG MARKER: == sanity-lnet test 629: lnet_legacy2yaml: multiple gateways error ========================================================== 22:12:10 (1781230330) [ 1159.342761] Lustre: DEBUG MARKER: == sanity-lnet test complete, duration 1032 sec ========== 22:12:13 (1781230333) [ 1159.501912] LNet: 139803:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1159.508966] LNetError: 139803:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1159.520246] LNet: Removed LNI 192.168.206.20@tcp [ 1159.806606] Key type .llcrypt unregistered [ 1159.808604] Key type ._llcrypt unregistered [ 1164.842357] Key type ._llcrypt registered [ 1164.844180] Key type .llcrypt registered [ 1165.024549] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 2 [ 1165.032593] alg: No test for adler32 (adler32-zlib) [ 1165.920310] Lustre: Lustre: Build Version: 2.17.53_61_g0d029bc [ 1166.040954] LNet: Added LNI 192.168.206.20@tcp [8/256/0/180] [ 1167.640646] Key type lgssc registered [ 1168.171429] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1198.028612] Lustre: Mounted lustre-client [ 1201.017279] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1204.816689] Lustre: Unmounted lustre-client [ 1218.821430] Key type lgssc unregistered [ 1218.972884] LNet: 142721:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1218.976729] LNetError: 142721:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1218.988145] LNet: Removed LNI 192.168.206.20@tcp [ 1219.341271] Key type .llcrypt unregistered [ 1219.343368] Key type ._llcrypt unregistered