[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 547384155 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5950 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1BB7 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1A53 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1AC7 000090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1B57 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1B8F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1a53-0xbffe1ac6] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1a52] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1ac7-0xbffe1b56] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1b57-0xbffe1b8e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1b8f-0xbffe1bb6] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.002463] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004017] kvm-guest: setup PV IPIs [ 0.006000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.006029] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.007016] pid_max: default: 32768 minimum: 301 [ 0.009112] LSM: Security Framework initializing [ 0.010041] Yama: becoming mindful. [ 0.011031] SELinux: Initializing. [ 0.012066] *** VALIDATE selinux *** [ 0.018787] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.021752] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.022155] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.023118] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.024131] *** VALIDATE tmpfs *** [ 0.025448] *** VALIDATE proc *** [ 0.027108] *** VALIDATE cgroup *** [ 0.028010] *** VALIDATE cgroup2 *** [ 0.030189] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.031144] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.032006] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.033033] Spectre V2 : User space: Vulnerable [ 0.034005] Speculative Store Bypass: Vulnerable [ 0.037011] debug: unmapping init [mem 0xffffffff88c59000-0xffffffff88c60fff] [ 0.039778] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.040604] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.041020] ... version: 2 [ 0.042009] ... bit width: 48 [ 0.043006] ... generic registers: 4 [ 0.044008] ... value mask: 0000ffffffffffff [ 0.045007] ... max period: 00007fffffffffff [ 0.046008] ... fixed-purpose events: 3 [ 0.047007] ... event mask: 000000070000000f [ 0.048298] rcu: Hierarchical SRCU implementation. [ 0.050380] smp: Bringing up secondary CPUs ... [ 0.051484] x86: Booting SMP configuration: [ 0.052018] .... node #0, CPUs: #1 #2 #3 [ 0.067181] smp: Brought up 1 node, 4 CPUs [ 0.069009] smpboot: Max logical packages: 1 [ 0.070010] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.136441] node 0 deferred pages initialised in 62ms [ 0.140144] devtmpfs: initialized [ 0.141000] x86/mm: Memory block size: 128MB [ 0.143000] gcov: version magic: 0x41383552 [ 0.147009] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.151074] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.152295] pinctrl core: initialized pinctrl subsystem [ 0.153228] [ 0.153708] ************************************************************* [ 0.154010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.155008] ** ** [ 0.156008] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.157008] ** ** [ 0.158008] ** This means that this kernel is built to expose internal ** [ 0.159009] ** IOMMU data structures, which may compromise security on ** [ 0.160009] ** your system. ** [ 0.161006] ** ** [ 0.162018] ** If you see this message and you are not debugging the ** [ 0.163008] ** kernel, report this immediately to your vendor! ** [ 0.164009] ** ** [ 0.165010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.166008] ************************************************************* [ 0.167971] NET: Registered protocol family 16 [ 0.168401] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.169041] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.170045] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.171517] cpuidle: using governor menu [ 0.172005] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.173000] PCI: Using configuration type 1 for base access [ 0.173000] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.183130] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.184024] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.187040] cryptd: max_cpu_qlen set to 1000 [ 0.189168] ACPI: Added _OSI(Module Device) [ 0.190010] ACPI: Added _OSI(Processor Device) [ 0.191012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.192012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.195569] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.198611] ACPI: Interpreter enabled [ 0.199062] ACPI: PM: (supports S0 S3 S4 S5) [ 0.200014] ACPI: Using IOAPIC for interrupt routing [ 0.201114] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.202403] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.211484] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.212038] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.213013] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.214095] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.216316] acpiphp: Slot [2] registered [ 0.217128] acpiphp: Slot [3] registered [ 0.218090] acpiphp: Slot [4] registered [ 0.219086] acpiphp: Slot [5] registered [ 0.220113] acpiphp: Slot [6] registered [ 0.221028] acpiphp: Slot [7] registered [ 0.222079] acpiphp: Slot [8] registered [ 0.223111] acpiphp: Slot [9] registered [ 0.224093] acpiphp: Slot [10] registered [ 0.225198] acpiphp: Slot [11] registered [ 0.226098] acpiphp: Slot [12] registered [ 0.227077] acpiphp: Slot [13] registered [ 0.228078] acpiphp: Slot [14] registered [ 0.229088] acpiphp: Slot [15] registered [ 0.230109] acpiphp: Slot [16] registered [ 0.231071] acpiphp: Slot [17] registered [ 0.232096] acpiphp: Slot [18] registered [ 0.233066] acpiphp: Slot [19] registered [ 0.234087] acpiphp: Slot [20] registered [ 0.235083] acpiphp: Slot [21] registered [ 0.236067] acpiphp: Slot [22] registered [ 0.237064] acpiphp: Slot [23] registered [ 0.238093] acpiphp: Slot [24] registered [ 0.239103] acpiphp: Slot [25] registered [ 0.240067] acpiphp: Slot [26] registered [ 0.241065] acpiphp: Slot [27] registered [ 0.242072] acpiphp: Slot [28] registered [ 0.243068] acpiphp: Slot [29] registered [ 0.244063] acpiphp: Slot [30] registered [ 0.245103] acpiphp: Slot [31] registered [ 0.246054] PCI host bridge to bus 0000:00 [ 0.247014] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.248016] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.249037] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.250027] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.251016] pci_bus 0000:00: root bus resource [mem 0x180000000-0x1ffffffff window] [ 0.252022] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.253158] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.256326] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.258979] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.264015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.267049] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.268013] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.269021] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.270023] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.271871] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.272673] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.278049] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.284082] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x1e0 took 11718 usecs [ 0.290689] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.305018] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.324018] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.342867] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.347557] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.366825] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.379021] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.408016] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.415000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.423020] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.430016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.448000] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.467379] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.479146] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.491035] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.512049] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 0.523000] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.538021] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.548018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.570019] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 0.583000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.590015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.601022] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.620017] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 0.633819] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.641019] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.660019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.680016] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 0.705070] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.707463] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.710304] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.713430] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.715165] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.720034] iommu: Default domain type: Passthrough [ 0.722487] SCSI subsystem initialized [ 0.724111] ACPI: bus type USB registered [ 0.725082] usbcore: registered new interface driver usbfs [ 0.726079] usbcore: registered new interface driver hub [ 0.727068] usbcore: registered new device driver usb [ 0.728212] pps_core: LinuxPPS API ver. 1 registered [ 0.730009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.731125] PTP clock support registered [ 0.733097] EDAC MC: Ver: 3.0.0 [ 0.734458] PCI: Using ACPI for IRQ routing [ 0.737244] NetLabel: Initializing [ 0.738010] NetLabel: domain hash size = 128 [ 0.740009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.742137] NetLabel: unlabeled traffic allowed by default [ 0.744142] vgaarb: loaded [ 0.746271] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.748013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.756619] clocksource: Switched to clocksource kvm-clock [ 0.879696] VFS: Disk quotas dquot_6.6.0 [ 0.881051] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.883436] *** VALIDATE ramfs *** [ 0.884615] *** VALIDATE hugetlbfs *** [ 0.886197] pnp: PnP ACPI init [ 0.888881] pnp: PnP ACPI: found 6 devices [ 0.906192] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.910922] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.913829] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.915773] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.917763] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.919658] pci_bus 0000:00: resource 8 [mem 0x180000000-0x1ffffffff window] [ 0.921974] NET: Registered protocol family 2 [ 0.924278] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.929253] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.932725] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.947416] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.953579] TCP: Hash tables configured (established 65536 bind 65536) [ 0.959159] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.964281] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.967434] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.971128] NET: Registered protocol family 1 [ 0.976286] RPC: Registered named UNIX socket transport module. [ 0.979741] RPC: Registered udp transport module. [ 0.982136] RPC: Registered tcp transport module. [ 0.983804] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.986040] NET: Registered protocol family 44 [ 0.989118] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.993670] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.995166] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.996768] PCI: CLS 0 bytes, default 64 [ 0.998318] Unpacking initramfs... [ 3.157709] debug: unmapping init [mem 0xffff8d1afcc54000-0xffff8d1afffbffff] [ 3.162456] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.164801] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.167358] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 3.964191] Initialise system trusted keyrings [ 3.965581] Key type blacklist registered [ 3.969336] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.981151] zbud: loaded [ 3.985872] *** VALIDATE nfs *** [ 3.988036] *** VALIDATE nfs4 *** [ 3.991456] pstore: using deflate compression [ 3.999382] Platform Keyring initialized [ 4.286343] NET: Registered protocol family 38 [ 4.287814] Key type asymmetric registered [ 4.292508] Asymmetric key parser 'x509' registered [ 4.298683] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 4.306429] io scheduler mq-deadline registered [ 4.312243] io scheduler kyber registered [ 4.313675] io scheduler bfq registered [ 4.317816] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.325403] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.332529] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.339266] ACPI: Power Button [PWRF] [ 4.532505] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 4.665562] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.995850] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 5.259877] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 5.636978] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 5.686313] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 5.725341] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 5.740442] Non-volatile memory driver v1.3 [ 5.742241] Linux agpgart interface v0.103 [ 5.802811] virtio_blk virtio1: [vda] 132968 512-byte logical blocks (68.1 MB/64.9 MiB) [ 5.806938] vda: detected capacity change from 0 to 68079616 [ 5.830726] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 5.833317] vdb: detected capacity change from 0 to 1073741824 [ 5.867620] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 5.871240] vdc: detected capacity change from 0 to 2621440000 [ 5.893913] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 5.900443] vdd: detected capacity change from 0 to 2621440000 [ 5.921672] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 5.924464] vde: detected capacity change from 0 to 4294967296 [ 5.942883] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 5.948938] vdf: detected capacity change from 0 to 4294967296 [ 5.954853] libphy: Fixed MDIO Bus: probed [ 5.967930] usbcore: registered new interface driver usbserial_generic [ 5.970123] usbserial: USB Serial support registered for generic [ 5.972287] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 5.976123] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 5.977667] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 5.983441] mousedev: PS/2 mouse device common for all mice [ 5.987383] rtc_cmos 00:05: RTC can wake from S4 [ 5.989933] rtc_cmos 00:05: registered as rtc0 [ 5.991433] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 6.000102] intel_pstate: CPU model not supported [ 6.001790] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 6.012264] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 6.018347] hid: raw HID events driver (C) Jiri Kosina [ 6.020214] usbcore: registered new interface driver usbhid [ 6.022115] usbhid: USB HID core driver [ 6.023319] drop_monitor: Initializing network drop monitor service [ 6.029081] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 6.031840] Initializing XFRM netlink socket [ 6.035633] NET: Registered protocol family 10 [ 6.041088] Segment Routing with IPv6 [ 6.046075] NET: Registered protocol family 17 [ 6.048104] mpls_gso: MPLS GSO support [ 6.057436] RAS: Correctable Errors collector initialized. [ 6.059408] AVX version of gcm_enc/dec engaged. [ 6.060893] AES CTR mode by8 optimization enabled [ 6.266598] sched_clock: Marking stable (6266575436, 0)->(7375282531, -1108707095) [ 6.275711] registered taskstats version 1 [ 6.281329] Loading compiled-in X.509 certificates [ 6.283696] zswap: loaded using pool lzo/zbud [ 6.333159] Key type big_key registered [ 6.359622] Key type encrypted registered [ 6.360865] ima: No TPM chip found, activating TPM-bypass! [ 6.368765] ima: Allocated hash algorithm: sha1 [ 6.370427] ima: No architecture policies found [ 6.373398] evm: Initialising EVM extended attributes: [ 6.374871] evm: security.selinux [ 6.375852] evm: security.ima [ 6.380354] evm: security.capability [ 6.381487] evm: HMAC attrs: 0x1 [ 6.389433] rtc_cmos 00:05: setting system clock to 2025-08-13 14:37:15 UTC (1755095835) [ 6.400425] debug: unmapping init [mem 0xffffffff89c03000-0xffffffff89dfffff] [ 6.405325] debug: unmapping init [mem 0xffffffff88982000-0xffffffff88c58fff] [ 6.413134] Write protecting the kernel read-only data: 28672k [ 6.418556] debug: unmapping init [mem 0xffffffff87003000-0xffffffff871fffff] [ 6.423266] debug: unmapping init [mem 0xffffffff87914000-0xffffffff879fffff] [ 6.479899] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 6.500168] systemd[1]: Detected virtualization kvm. [ 6.505166] systemd[1]: Detected architecture x86-64. [ 6.511310] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 6.574409] systemd[1]: No hostname configured. [ 6.578667] systemd[1]: Set hostname to . [ 6.581712] random: systemd: uninitialized urandom read (16 bytes read) [ 6.586547] systemd[1]: Initializing machine ID from random generator. [ 6.953974] random: systemd: uninitialized urandom read (16 bytes read) [ 6.957874] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 6.969604] random: systemd: uninitialized urandom read (16 bytes read) [ 6.976209] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 6.982982] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket. Starting Journal Service... Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Timers. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 8.423116] device-mapper: uevent: version 1.0.3 [ 8.429699] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 9.949766] virtio_net virtio0 ens2: renamed from eth0 [ 10.237187] random: fast init done [ 10.429583] scsi host0: ata_piix [ 10.452335] scsi host1: ata_piix [ 10.453601] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 10.455752] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 15.929917] random: crng init done [ 15.934560] random: 7 urandom warning(s) missed due to ratelimiting [ 18.479301] dracut-initqueue[590]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 19.912699] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 23.551597] printk: systemd: 26 output lines suppressed due to ratelimiting [ 24.324170] SELinux: Disabled at runtime. [ 24.438277] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 24.455453] systemd[1]: Detected virtualization kvm. [ 24.457259] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 26.212107] systemd[1]: initrd-switch-root.service: Succeeded. [ 26.221300] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 26.243287] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 26.275211] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 26.278311] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 26.289483] systemd[1]: Starting Journal Service... Starting Journal Service... [ 26.328281] systemd[1]: Stopped target Switch Root. [ OK ] Stopped target Switch Root. [ OK ] Listening on udev Control Socket. Mounting Huge Pages File System... [ OK ] Reached target rpc_pipefs.target. Mounting POSIX Message Queue File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on RPCbind Server Activation Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target RPC Port Mapper. [ 28.086084] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-serial\x2dgetty.slice. Starting Apply Kernel Variables... [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice system-getty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. Starting Remount Root and Kernel File Systems... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Kernel Debug File System... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 29.186188] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 30.198272] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 30.202172] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 31.055142] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 31.179258] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit) [** ] A start job is running for Configur…-only root support (9s / no limit)[ 35.572474] Key type dns_resolver registered [*** ] A start job is running for Configur…-only root support (9s / no limit)[ 35.901020] hrtimer: interrupt took 8012170 ns [ *** ] A start job is running for Configur…only root support (10s / no limit)[ 36.454385] NFS: Registering the id_resolver key type [ 36.456833] Key type id_resolver registered [ 36.458597] Key type id_legacy registered [ *** ] A start job is running for Configur…only root support (10s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Reached target sshd-keygen.target. Starting Network Manager... Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg344-server login: [ 73.867899] libcfs: loading out-of-tree module taints kernel. [ 73.878950] Key type ._llcrypt registered [ 73.880534] Key type .llcrypt registered [ 73.917425] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 81.614810] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 82.211695] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 82.218514] alg: No test for adler32 (adler32-zlib) [ 83.220436] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 83.536196] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 83.538395] LNet: Accept secure, port 988 [ 85.167322] Key type lgssc registered [ 85.792843] Lustre: Echo OBD driver; http://www.lustre.org/ [ 93.220968] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 94.319094] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 98.220931] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 101.037730] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 104.182102] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 110.304982] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 115.178229] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 115.201907] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 115.217868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 116.327842] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 116.346540] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 116.389267] Lustre: lustre-MDT0000: new disk, initializing [ 116.420595] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 116.429101] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 118.009104] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 124.149072] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 124.184587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 124.223528] Lustre: 6454:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 124.239836] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 124.242517] Lustre: Skipped 1 previous similar message [ 124.288231] Lustre: lustre-MDT0001: new disk, initializing [ 124.317849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 124.331782] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 124.337404] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 125.827973] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 128.249683] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 131.690877] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 131.716229] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 131.803652] Lustre: lustre-OST0000: new disk, initializing [ 131.805297] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 131.822556] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 133.951967] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 139.252182] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 139.255309] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 139.281947] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 140.068380] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 140.108269] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 140.164578] Lustre: lustre-OST0001: new disk, initializing [ 140.168294] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 140.201239] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 142.560731] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 148.863717] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 148.972405] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 148.976287] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 148.994946] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 152.423252] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 159.600210] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing check_logdir /tmp/testlogs/ [ 161.215821] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing yml_node [ 162.930685] Lustre: DEBUG MARKER: Client: 2.16.56.9 [ 163.813699] Lustre: DEBUG MARKER: MDS: 2.16.56.9 [ 164.729903] Lustre: DEBUG MARKER: OSS: 2.16.56.9 [ 165.291520] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Aug 13 10:39:53 EDT 2025 [ 171.851401] Lustre: DEBUG MARKER: excepting tests: 55 63 67 32newtarball [ 172.460085] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 184.803243] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 184.805591] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 184.808823] Lustre: Skipped 1 previous similar message [ 189.921069] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 189.925048] Lustre: Skipped 6 previous similar messages [ 190.591993] LustreError: 12119:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 190.637032] Lustre: server umount lustre-MDT0000 complete [ 192.157928] LustreError: 12020:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096021 with bad export cookie 4077718128931272847 [ 192.159801] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 192.162793] LustreError: 12020:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 195.040586] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 195.041599] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 195.046656] Lustre: Skipped 3 previous similar messages [ 197.088464] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 197.091221] Lustre: Skipped 2 previous similar messages [ 198.278512] LustreError: 12322:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 198.280697] LustreError: 12322:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 198.377513] Lustre: server umount lustre-MDT0001 complete [ 200.277246] LustreError: 12523:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 200.279271] LustreError: 12523:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 200.308094] Lustre: server umount lustre-OST0000 complete [ 202.160122] Lustre: server umount lustre-OST0001 complete [ 204.855960] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 208.542907] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 212.678873] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 215.601934] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 218.192668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 220.919333] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 223.883193] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 223.908375] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 224.008137] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 224.023357] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 224.065760] Lustre: lustre-MDT0000: new disk, initializing [ 224.101843] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 224.111135] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 225.757285] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 230.265397] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 230.293038] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 230.328974] Lustre: 14886:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 230.431563] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 231.784966] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 234.886406] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 236.123857] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 238.471993] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 238.495155] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 238.594537] Lustre: lustre-OST0000: new disk, initializing [ 238.596242] Lustre: Skipped 1 previous similar message [ 238.598262] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 238.601388] Lustre: Skipped 2 previous similar messages [ 238.633388] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 238.636184] Lustre: Skipped 1 previous similar message [ 239.735078] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 239.737697] Lustre: Skipped 1 previous similar message [ 239.739195] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 239.774054] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 240.435123] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 243.577505] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 244.982485] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 245.061133] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 246.292569] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 246.360313] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 249.006080] LustreError: 17793:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 249.008889] LustreError: 17793:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 249.070532] Lustre: server umount lustre-OST0000 complete [ 250.336257] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 250.336670] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 250.341564] Lustre: Skipped 1 previous similar message [ 260.576899] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 260.578727] Lustre: Skipped 4 previous similar messages [ 260.580660] LustreError: 17993:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 260.582374] LustreError: 17993:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 260.600990] Lustre: server umount lustre-MDT0000 complete [ 261.808321] LustreError: 14878:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096090 with bad export cookie 4077718128931274387 [ 261.810743] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 261.812783] LustreError: 14878:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 271.331930] LustreError: 18196:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 271.333845] LustreError: 18196:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 271.421050] Lustre: server umount lustre-MDT0001 complete [ 272.873085] Lustre: DEBUG MARKER: == conf-sanity test 33a: Mount ost with a large index number ========================================================== 10:41:40 (1755096100) [ 273.700513] Lustre: DEBUG MARKER: SKIP: conf-sanity test_33a mixed loopback and real device not working [ 274.253599] Lustre: DEBUG MARKER: == conf-sanity test 33b: Drop cancel during umount ======= 10:41:42 (1755096102) [ 276.558669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 276.724117] LustreError: 18866:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 276.730481] LustreError: 18866:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 5 previous similar messages [ 276.749455] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 278.082387] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 281.012772] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 282.529504] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 284.323955] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 285.478628] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 287.710812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 289.600996] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 291.316672] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 311.776609] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 311.778876] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 311.783314] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 312.801117] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 315.694695] LustreError: 21110:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 315.696994] LustreError: 21110:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 315.763141] Lustre: server umount lustre-OST0000 complete [ 317.920934] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 317.925584] Lustre: Skipped 2 previous similar messages [ 323.226343] Lustre: server umount lustre-MDT0000 complete [ 324.491520] LustreError: 18845:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096153 with bad export cookie 4077718128931275269 [ 324.495715] LustreError: 18845:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 1 previous similar message [ 324.496871] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 324.656040] Lustre: server umount lustre-MDT0001 complete [ 329.283345] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 330.333237] Key type lgssc unregistered [ 330.471374] LNet: 22041:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 331.493307] LNet: Removed LNI 192.168.203.144@tcp [ 331.823333] Key type .llcrypt unregistered [ 331.824760] Key type ._llcrypt unregistered [ 340.333821] Key type ._llcrypt registered [ 340.335381] Key type .llcrypt registered [ 340.377944] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 340.800870] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 340.808560] alg: No test for adler32 (adler32-zlib) [ 341.678990] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 341.777028] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 341.779809] LNet: Accept secure, port 988 [ 343.367137] Key type lgssc registered [ 343.731420] Lustre: Echo OBD driver; http://www.lustre.org/ [ 346.422210] loop: module loaded [ 346.442285] loop0: detected capacity change from 0 to 204800000 [ 346.550813] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 346.554105] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 346.676210] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 346.681636] blk_update_request: operation not supported error, dev loop0, sector 160240 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 346.708323] blk_update_request: operation not supported error, dev loop0, sector 160344 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 346.732262] blk_update_request: operation not supported error, dev loop0, sector 168552 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 346.750125] blk_update_request: operation not supported error, dev loop0, sector 173280 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 346.777113] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 348.663887] loop0: detected capacity change from 0 to 204800000 [ 348.726052] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 348.729850] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 348.733129] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 348.785092] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 350.388357] loop0: detected capacity change from 0 to 204800000 [ 352.229811] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 352.249771] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 352.254906] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 353.332114] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 353.345931] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 353.384689] Lustre: lustre-MDT0000: new disk, initializing [ 353.408166] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 353.415098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 354.687442] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 358.281774] loop1: detected capacity change from 0 to 204800000 [ 360.265791] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 360.292908] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 360.409080] Lustre: lustre-OST07c6: new disk, initializing [ 360.411927] Lustre: srv-lustre-OST07c6: No data found on store. Initialize space. [ 360.412423] Lustre: lustre-OST07c6: Not available for connect from 0@lo (not set up) [ 360.414496] Lustre: Skipped 1 previous similar message [ 360.439693] Lustre: lustre-OST07c6: Imperative Recovery not enabled, recovery window 60-180 [ 362.373310] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 365.550087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:7c6:ost [ 365.554314] Lustre: cli-lustre-OST07c6-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:7c6:ost] [ 365.566043] Lustre: lustre-OST07c6-osc-MDT0000: update sequence from 0x107c60000 to 0x240000400 [ 452.577833] Lustre: lustre-OST07c6-osc-MDT0000: Connection to lustre-OST07c6 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 452.577878] LustreError: lustre-OST07c6-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 452.583162] Lustre: lustre-OST07c6: Not available for connect from 0@lo (stopping) [ 455.722648] LustreError: 26365:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 455.791135] Lustre: server umount lustre-OST07c6 complete [ 459.078841] LustreError: 26865:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 459.081556] LustreError: 26865:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 459.166896] Lustre: server umount lustre-MDT0000 complete [ 465.810507] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 466.942872] Key type lgssc unregistered [ 467.090406] LNet: 27691:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 468.133332] LNet: Removed LNI 192.168.203.144@tcp [ 468.423243] Key type .llcrypt unregistered [ 468.424486] Key type ._llcrypt unregistered [ 476.277505] Key type ._llcrypt registered [ 476.278647] Key type .llcrypt registered [ 476.321554] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 476.645059] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 476.657265] alg: No test for adler32 (adler32-zlib) [ 477.530568] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 477.623036] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 477.625206] LNet: Accept secure, port 988 [ 479.215157] Key type lgssc registered [ 479.703065] Lustre: Echo OBD driver; http://www.lustre.org/ [ 482.701174] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 482.712699] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 483.906661] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 485.378536] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 488.547335] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 488.679024] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 490.151953] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 492.237209] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 493.510925] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 496.041670] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 496.177952] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 498.262061] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 500.257926] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 508.451512] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:35 to 0x280000401:65) [ 513.549988] LustreError: 31105:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:2 until pending updates are flushed. [ 518.624502] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 518.627029] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 518.628469] LustreError: Skipped 1 previous similar message [ 518.633829] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 518.637428] Lustre: Skipped 1 previous similar message [ 522.413495] LustreError: 31357:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 522.479233] Lustre: server umount lustre-OST0000 complete [ 524.767850] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 524.770230] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 524.775670] Lustre: Skipped 1 previous similar message [ 524.778153] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 528.863933] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 528.864713] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 530.110405] LustreError: 31559:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 530.112430] LustreError: 31559:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 530.137946] Lustre: server umount lustre-MDT0000 complete [ 531.510296] LustreError: 28890:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096360 with bad export cookie 14367915640889294808 [ 531.514184] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 531.515236] LustreError: 28890:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 531.591803] LustreError: 31761:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 531.594092] LustreError: 31761:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 531.680533] Lustre: server umount lustre-MDT0001 complete [ 536.398429] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 537.551527] Key type lgssc unregistered [ 537.687388] LNet: 32289:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 538.725456] LNet: Removed LNI 192.168.203.144@tcp [ 539.052554] Key type .llcrypt unregistered [ 539.053894] Key type ._llcrypt unregistered [ 546.864480] Key type ._llcrypt registered [ 546.865863] Key type .llcrypt registered [ 546.911820] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 547.271866] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 547.285061] alg: No test for adler32 (adler32-zlib) [ 548.138045] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 548.224698] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 548.226375] LNet: Accept secure, port 988 [ 549.807272] Key type lgssc registered [ 550.192471] Lustre: Echo OBD driver; http://www.lustre.org/ [ 553.004824] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 553.015268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 554.200202] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 555.618781] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 558.809541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 558.947973] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 560.416582] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 562.388231] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 563.666594] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 566.092859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 566.224868] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 568.210105] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 570.131236] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 579.568335] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:67 to 0x280000401:97) [ 589.795112] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 589.795452] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 589.804383] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 589.806830] Lustre: Skipped 1 previous similar message [ 594.916295] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 594.919226] Lustre: Skipped 1 previous similar message [ 599.007088] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 599.022018] LustreError: 35753:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 599.076832] Lustre: server umount lustre-OST0000 complete [ 605.153873] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 605.154165] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 605.154827] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 605.159653] Lustre: Skipped 2 previous similar messages [ 605.165308] LustreError: Skipped 1 previous similar message [ 606.459988] LustreError: 35955:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 606.461537] LustreError: 35955:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 606.482382] Lustre: server umount lustre-MDT0000 complete [ 607.738552] LustreError: 33487:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096436 with bad export cookie 9750210347193207489 [ 607.741487] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 607.742987] LustreError: 33487:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 607.802130] LustreError: 36157:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 607.804677] LustreError: 36157:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 607.879984] Lustre: server umount lustre-MDT0001 complete [ 612.610073] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 613.693326] Key type lgssc unregistered [ 613.841443] LNet: 36683:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 614.885457] LNet: Removed LNI 192.168.203.144@tcp [ 615.203724] Key type .llcrypt unregistered [ 615.205150] Key type ._llcrypt unregistered [ 624.068488] Key type ._llcrypt registered [ 624.070071] Key type .llcrypt registered [ 624.108258] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 624.531718] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 624.539495] alg: No test for adler32 (adler32-zlib) [ 625.410920] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 625.499521] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 625.501470] LNet: Accept secure, port 988 [ 627.087155] Key type lgssc registered [ 627.454590] Lustre: Echo OBD driver; http://www.lustre.org/ [ 630.031202] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 630.037024] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 631.199894] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 632.557855] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 635.347538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 635.472564] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 636.769357] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 638.550351] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 639.719478] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 642.025060] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 642.132527] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 643.997032] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 645.756732] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 656.485958] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:99 to 0x280000401:129) [ 660.960601] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 660.963105] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 660.966845] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 661.620383] Lustre: lustre-MDT0000: Not available for connect from 192.168.203.44@tcp (stopping) [ 661.986382] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 661.990352] Lustre: Skipped 1 previous similar message [ 664.891509] LustreError: 40294:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 664.920948] Lustre: server umount lustre-MDT0000 complete [ 666.206203] LustreError: 38026:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096495 with bad export cookie 12896792830464628523 [ 666.209708] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 666.211109] LustreError: 38026:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 666.732987] LustreError: 38070:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.44@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 666.739430] LustreError: 38070:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 666.742610] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.44@tcp (stopping) [ 666.744962] Lustre: Skipped 3 previous similar messages [ 667.104333] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 671.135783] LustreError: 38041:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 671.136078] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 671.141249] LustreError: 38041:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 671.143682] Lustre: Skipped 1 previous similar message [ 672.319577] LustreError: 40497:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 672.321934] LustreError: 40497:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 672.404482] Lustre: server umount lustre-MDT0001 complete [ 690.346877] LustreError: 40722:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 690.349017] LustreError: 40722:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 690.419631] Lustre: server umount lustre-OST0000 complete [ 697.172409] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 698.325582] Key type lgssc unregistered [ 698.463517] LNet: 41553:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 699.493534] LNet: Removed LNI 192.168.203.144@tcp [ 699.830205] Key type .llcrypt unregistered [ 699.831526] Key type ._llcrypt unregistered [ 708.804958] Key type ._llcrypt registered [ 708.806456] Key type .llcrypt registered [ 708.849243] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 709.190136] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 709.217113] alg: No test for adler32 (adler32-zlib) [ 710.083472] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 710.170594] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 710.172680] LNet: Accept secure, port 988 [ 711.759148] Key type lgssc registered [ 712.118021] Lustre: Echo OBD driver; http://www.lustre.org/ [ 714.726746] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 714.733045] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 715.893187] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 717.277165] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 720.288627] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 720.420143] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 721.813158] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 723.750322] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 725.127373] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 727.412209] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 727.532605] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 729.522107] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 731.436786] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 741.866756] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:131 to 0x280000401:161) [ 745.952899] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 745.955592] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 745.960049] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 746.978078] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 746.979252] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 746.979628] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 746.988309] Lustre: Skipped 1 previous similar message [ 750.315129] LustreError: 45177:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 750.392350] Lustre: server umount lustre-OST0000 complete [ 763.121136] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 763.122051] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 763.125940] Lustre: Skipped 1 previous similar message [ 763.129743] Lustre: Skipped 2 previous similar messages [ 768.480584] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 768.482463] Lustre: Skipped 1 previous similar message [ 769.341753] LustreError: 45554:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 769.344800] LustreError: 45554:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 769.369075] Lustre: server umount lustre-MDT0000 complete [ 770.654130] LustreError: 45298:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096599 with bad export cookie 9349924345907397834 [ 770.658267] LustreError: 45298:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 770.658412] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 776.764576] LustreError: 45755:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 776.766230] LustreError: 45755:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 776.848813] Lustre: server umount lustre-MDT0001 complete [ 781.498505] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 782.613504] Key type lgssc unregistered [ 782.751382] LNet: 46281:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 783.782108] LNet: Removed LNI 192.168.203.144@tcp [ 784.124393] Key type .llcrypt unregistered [ 784.125630] Key type ._llcrypt unregistered [ 792.877696] Key type ._llcrypt registered [ 792.878865] Key type .llcrypt registered [ 792.916219] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 793.213893] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 793.230550] alg: No test for adler32 (adler32-zlib) [ 794.090948] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 794.171477] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 794.175025] LNet: Accept secure, port 988 [ 795.767265] Key type lgssc registered [ 796.161536] Lustre: Echo OBD driver; http://www.lustre.org/ [ 798.859601] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 798.868317] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 800.030697] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 801.328880] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 804.093100] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 804.228920] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 805.519584] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 807.379642] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 808.634289] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 810.866749] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 810.985406] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 812.871303] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 814.675909] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 825.322392] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:163 to 0x280000401:193) [ 827.470611] Lustre: DEBUG MARKER: Set up a fake failnode for the MDS [ 828.517640] Lustre: DEBUG MARKER: Wait for RECONNECT_INTERVAL seconds (10s) [ 834.022964] LNetError: Refusing connection from 192.168.203.144 for 127.0.0.2@tcp: No matching NI [ 834.025549] LNetError: 46948:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 834.028228] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 839.228669] Lustre: DEBUG MARKER: conf-sanity.sh test_35a 2025-08-1310h51m06s [ 839.729867] Lustre: DEBUG MARKER: Stopping the MDT: lustre-MDT0000 [ 840.672712] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 840.673638] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 840.676617] Lustre: Skipped 1 previous similar message [ 840.680287] LNetError: Refusing connection from 192.168.203.144 for 127.0.0.2@tcp: No matching NI [ 840.683314] LNetError: 46951:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 840.685950] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 841.697394] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 841.697675] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 841.703654] Lustre: Skipped 4 previous similar messages [ 842.720105] LNetError: Refusing connection from 192.168.203.144 for 127.0.0.2@tcp: No matching NI [ 842.721820] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 842.722956] LNetError: 46949:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 842.724675] Lustre: Skipped 1 previous similar message [ 842.729455] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 844.770113] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 844.772114] Lustre: Skipped 2 previous similar messages [ 846.589886] LustreError: 50208:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 846.618592] Lustre: server umount lustre-MDT0000 complete [ 846.817102] LustreError: 47644:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 846.817768] LNetError: Refusing connection from 192.168.203.144 for 127.0.0.2@tcp: No matching NI [ 846.821791] LustreError: 47644:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 846.825085] LNetError: 46950:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 846.830152] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 847.743917] Lustre: DEBUG MARKER: Restarting the MDT: lustre-MDT0000 [ 847.840707] LustreError: 47639:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 847.845868] LustreError: 47639:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 848.865327] LustreError: 47644:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 848.870640] LustreError: 47644:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 849.962344] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 849.999643] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 850.089035] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 850.114464] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:163 to 0x280000401:225) [ 851.109458] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to (at 0@lo) [ 851.382457] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 852.558060] Lustre: DEBUG MARKER: Wait for df (35896) ... [ 853.085450] Lustre: DEBUG MARKER: done [ 855.007739] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 855.010034] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 855.013809] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 855.015394] Lustre: Skipped 2 previous similar messages [ 855.520282] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 856.107124] LustreError: 51251:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 856.108921] LustreError: 51251:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 856.169963] Lustre: server umount lustre-OST0000 complete [ 859.514611] LustreError: 51456:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 859.516376] LustreError: 51456:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 859.540194] Lustre: server umount lustre-MDT0000 complete [ 860.766983] LustreError: 49250:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096689 with bad export cookie 15105881315659793308 [ 860.769726] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 860.770421] LustreError: 49250:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 866.875062] LustreError: 51656:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 866.877436] LustreError: 51656:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 866.953546] Lustre: server umount lustre-MDT0001 complete [ 871.380286] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 872.414513] Key type lgssc unregistered [ 872.546325] LNet: 52183:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 873.573295] LNet: Removed LNI 192.168.203.144@tcp [ 873.865854] Key type .llcrypt unregistered [ 873.866865] Key type ._llcrypt unregistered [ 884.771871] Key type ._llcrypt registered [ 884.773115] Key type .llcrypt registered [ 884.806777] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 890.730955] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 891.177850] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 891.189232] alg: No test for adler32 (adler32-zlib) [ 892.032886] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 892.111212] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 892.113226] LNet: Accept secure, port 988 [ 893.687149] Key type lgssc registered [ 894.011045] Lustre: Echo OBD driver; http://www.lustre.org/ [ 896.673629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 898.681315] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 900.610635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 902.615449] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 904.814816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 904.832669] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 904.838720] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 905.917950] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 905.927687] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 905.960388] Lustre: lustre-MDT0000: new disk, initializing [ 905.980505] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 905.986272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 907.188946] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 911.381151] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 911.405567] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 911.444947] Lustre: 56087:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 911.468841] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 911.471857] Lustre: Skipped 1 previous similar message [ 911.515825] Lustre: lustre-MDT0001: new disk, initializing [ 911.552442] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 911.566640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 911.569400] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 912.952451] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 916.093568] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 917.267825] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 919.617697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 919.641180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 919.741403] Lustre: lustre-OST0000: new disk, initializing [ 919.743826] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 919.772462] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 921.794711] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 924.668804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 924.672603] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 924.712328] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 924.819348] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 926.207814] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 926.291744] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 927.601991] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 927.675877] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 929.766545] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 929.766608] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 929.771247] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 929.775537] Lustre: Skipped 1 previous similar message [ 929.778837] Lustre: Skipped 1 previous similar message [ 934.381146] LustreError: 58997:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 934.451462] Lustre: server umount lustre-OST0000 complete [ 937.439811] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 937.442833] LustreError: Skipped 1 previous similar message [ 937.444018] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 937.447420] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 940.000148] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 940.000564] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 945.119995] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 945.121672] Lustre: Skipped 2 previous similar messages [ 949.216438] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 949.218674] Lustre: Skipped 1 previous similar message [ 950.239119] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 950.273835] LustreError: 59198:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 950.276038] LustreError: 59198:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 950.303625] Lustre: server umount lustre-MDT0000 complete [ 951.498987] LustreError: 56861:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096780 with bad export cookie 2950998263763959761 [ 951.502218] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 951.503186] LustreError: 56861:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 951.568676] LustreError: 59400:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 951.571795] LustreError: 59400:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 951.650072] Lustre: server umount lustre-MDT0001 complete [ 954.297962] Lustre: DEBUG MARKER: == conf-sanity test 35b: Continue reconnection retries, if the active server is busy ========================================================== 10:53:02 (1755096782) [ 956.387362] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 956.538819] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 957.754531] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 960.484057] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 960.607920] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 961.950689] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 963.701777] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 964.867827] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 967.133016] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 969.014658] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 970.686552] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 988.553333] Lustre: DEBUG MARKER: conf-sanity.sh test_35b 2025-08-1310h53m36s [ 989.101149] Lustre: DEBUG MARKER: Set up a fake failnode for the MDS [ 991.442286] Lustre: DEBUG MARKER: Injecting EBUSY on MDS [ 992.229938] Lustre: DEBUG MARKER: Creating a test file and stat it [ 992.248251] LustreError: 60785:0:(mdt_handler.c:2326:mdt_getattr_name_lock()) cfs_fail_timeout id 136 sleeping for 40000ms [ 999.907634] LNetError: Refusing connection from 192.168.203.144 for 127.0.0.2@tcp: No matching NI [ 999.910440] LNetError: 54295:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 999.915226] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 1012.344294] Lustre: lustre-MDT0000: Client 6237c723-f10d-4efd-b6fb-c9ef70b61850 (at 192.168.203.44@tcp) reconnecting [ 1032.295118] LustreError: 60785:0:(mdt_handler.c:2326:mdt_getattr_name_lock()) cfs_fail_timeout id 136 awake [ 1032.973830] Lustre: DEBUG MARKER: Stop injecting EBUSY on MDS [ 1033.785662] Lustre: DEBUG MARKER: done [ 1037.280170] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1037.282857] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1037.287278] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1041.387073] LustreError: 63076:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1041.388823] LustreError: 63076:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1041.441207] Lustre: server umount lustre-OST0000 complete [ 1044.448598] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1044.451912] Lustre: Skipped 2 previous similar messages [ 1044.453597] LNetError: Refusing connection from 192.168.203.144 for 127.0.0.2@tcp: No matching NI [ 1044.455695] LNetError: 54298:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 1044.459136] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 1048.635740] LustreError: 63277:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1048.637321] LustreError: 63277:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1048.659246] Lustre: server umount lustre-MDT0000 complete [ 1049.504910] LNetError: Refusing connection from 192.168.203.144 for 127.0.0.2@tcp: No matching NI [ 1049.507261] LNetError: 54296:0:(socklnd_cb.c:1794:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 1049.509347] LNetError: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 1049.767173] LustreError: 62463:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096878 with bad export cookie 2950998263763960629 [ 1049.769992] LustreError: 62463:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1049.770312] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1049.910612] Lustre: server umount lustre-MDT0001 complete [ 1054.450101] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 1055.517494] Key type lgssc unregistered [ 1055.649337] LNet: 64005:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1056.678250] LNet: Removed LNI 192.168.203.144@tcp [ 1056.982706] Key type .llcrypt unregistered [ 1056.983611] Key type ._llcrypt unregistered [ 1067.045814] Key type ._llcrypt registered [ 1067.046690] Key type .llcrypt registered [ 1067.083559] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 1072.941600] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 1073.249822] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1073.296929] alg: No test for adler32 (adler32-zlib) [ 1074.145572] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 1074.218602] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 1074.220197] LNet: Accept secure, port 988 [ 1075.799185] Key type lgssc registered [ 1076.094095] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1078.803760] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1080.837764] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1082.780605] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1084.756411] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1086.829920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1086.843988] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1086.849671] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1087.929093] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1087.939564] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1087.971731] Lustre: lustre-MDT0000: new disk, initializing [ 1087.991844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1087.997932] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1089.249967] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1093.111016] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1093.132441] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1093.167247] Lustre: 67908:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1093.198671] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1093.200845] Lustre: Skipped 1 previous similar message [ 1093.233437] Lustre: lustre-MDT0001: new disk, initializing [ 1093.262548] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1093.274293] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1093.278553] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1094.410222] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1097.348220] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1098.459826] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1100.574533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1100.595893] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1100.687044] Lustre: lustre-OST0000: new disk, initializing [ 1100.688975] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1100.690945] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 1100.726186] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1102.512059] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1105.390491] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1105.895726] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1105.898523] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1105.908811] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1106.595624] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1106.658992] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1107.776183] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1107.850199] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1111.008629] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1111.010822] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1111.011933] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1111.014736] Lustre: Skipped 1 previous similar message [ 1111.018952] Lustre: Skipped 1 previous similar message [ 1114.475076] LustreError: 70818:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1114.540223] Lustre: server umount lustre-OST0000 complete [ 1116.129571] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1116.130683] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1116.134307] Lustre: Skipped 1 previous similar message [ 1121.248295] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1121.251345] Lustre: Skipped 2 previous similar messages [ 1121.918913] LustreError: 71024:0:(obd_class.h:479:obd_check_dev()) Device 20 not setup [ 1121.921316] LustreError: 71024:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1121.947876] Lustre: server umount lustre-MDT0000 complete [ 1123.266237] LustreError: 67900:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096952 with bad export cookie 17248622621884320497 [ 1123.268076] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1123.269770] LustreError: 67900:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1123.332474] LustreError: 71226:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1123.336091] LustreError: 71226:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1123.413615] Lustre: server umount lustre-MDT0001 complete [ 1126.180410] Lustre: DEBUG MARKER: == conf-sanity test 36: df report consistency on OSTs with different block size ========================================================== 10:55:54 (1755096954) [ 1126.700170] Lustre: DEBUG MARKER: SKIP: conf-sanity test_36 remote OST [ 1127.331405] Lustre: DEBUG MARKER: == conf-sanity test 37: verify set tunables works for symlink device ========================================================== 10:55:55 (1755096955) [ 1131.427598] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 1132.924822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1133.079576] LustreError: 72261:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1133.085892] LustreError: 72261:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 1133.096417] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1133.470572] Lustre: Failing over lustre-MDT0000 [ 1133.473037] LustreError: 72290:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1133.477152] LustreError: 72290:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1133.480051] LustreError: 72290:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 0, retries 0, failed: rc = -5 [ 1133.551207] LustreError: 72341:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1133.553909] LustreError: 72341:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1133.651191] Lustre: server umount lustre-MDT0000 complete [ 1135.786482] Lustre: DEBUG MARKER: == conf-sanity test 38: MDS recreates missing lov_objid file from OST data ========================================================== 10:56:03 (1755096963) [ 1137.868270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1137.989021] LustreError: 72909:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1138.004295] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1139.143835] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1141.775060] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1143.103548] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1144.809498] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1145.893553] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1147.962276] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1148.066993] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1148.070108] Lustre: Skipped 1 previous similar message [ 1149.849408] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1151.473818] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1154.454324] Lustre: DEBUG MARKER: copying 10 files to /mnt/lustre/d38.conf-sanity [ 1156.577113] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1156.578472] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1156.581401] Lustre: Skipped 2 previous similar messages [ 1156.583854] Lustre: Skipped 2 previous similar messages [ 1162.105064] LustreError: 75233:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1162.107290] LustreError: 75233:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1162.128465] Lustre: server umount lustre-MDT0000 complete [ 1163.229946] LustreError: 72933:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755096992 with bad export cookie 17248622621884321652 [ 1163.232454] LustreError: 72933:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1163.232478] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1163.355941] Lustre: server umount lustre-MDT0001 complete [ 1164.403726] Lustre: DEBUG MARKER: delete lov_objid file on MDS [ 1165.935677] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1168.801503] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1172.959595] LustreError: 66128:0:(client.c:1385:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000bf1e30ef x1840352490216192/t0(0) o250->MGC192.168.203.144@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1173.086903] LustreError: 74504:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1173.087913] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1173.093735] LustreError: 74504:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 1173.127619] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1173.168440] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000402 [ 1174.483293] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1177.360638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1178.862546] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1180.747945] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1181.985542] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1182.623148] Lustre: 66132:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755096995/real 1755096995] req@00000000793c795a x1840352490215808/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1755097011 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1182.692970] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to (at 0@lo) [ 1184.735160] Lustre: 66130:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755096997/real 1755096997] req@00000000a4c07939 x1840352490215936/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1755097013 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1186.792838] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to (at 0@lo) [ 1187.807302] Lustre: 66130:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755097000/real 1755097000] req@00000000659d17b0 x1840352490216064/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1755097016 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1189.983128] Lustre: 66132:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755097002/real 1755097002] req@000000005a166abb x1840352490216320/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1755097018 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1191.905550] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1191.906613] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1191.909473] Lustre: Skipped 2 previous similar messages [ 1191.911492] Lustre: Skipped 4 previous similar messages [ 1195.063271] LustreError: 77681:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1195.064685] LustreError: 77681:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 1195.084734] Lustre: server umount lustre-MDT0000 complete [ 1196.247187] LustreError: 72891:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097025 with bad export cookie 17248622621884323542 [ 1196.250190] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1196.251795] LustreError: 72891:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1196.384778] Lustre: server umount lustre-MDT0001 complete [ 1198.398351] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1201.328753] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1206.175901] LustreError: 66128:0:(client.c:1385:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000ff55ef99 x1840352490236032/t0(0) o250->MGC192.168.203.144@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1206.294597] LustreError: 77014:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1206.321572] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1206.324910] Lustre: Skipped 1 previous similar message [ 1206.347473] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000402:3 to 0x280000402:33) [ 1207.737705] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1210.558155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1210.637441] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1211.797271] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1212.708192] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to (at 0@lo) [ 1213.338319] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1213.728072] Lustre: 66131:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755097026/real 1755097026] req@000000008d1c675a x1840352490235648/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1755097042 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1214.571863] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1218.019088] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1218.020980] Lustre: Skipped 2 previous similar messages [ 1222.111135] Lustre: 66132:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755097035/real 1755097035] req@00000000c2921a0d x1840352490236160/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1755097051 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1222.118900] Lustre: 66132:0:(client.c:2453:ptlrpc_expire_one_request()) Skipped 2 previous similar messages [ 1222.463799] LustreError: 80126:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1222.466668] LustreError: 80126:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 1222.498256] Lustre: server umount lustre-MDT0000 complete [ 1223.138160] LustreError: 74504:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1223.144304] LustreError: 74504:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 1223.818255] LustreError: 72933:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097052 with bad export cookie 17248622621884324914 [ 1223.821376] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1223.823131] LustreError: 72933:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1223.966763] Lustre: server umount lustre-MDT0001 complete [ 1226.237730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1227.285593] Lustre: DEBUG MARKER: files compared the same [ 1237.160473] Lustre: server umount lustre-OST0000 complete [ 1243.509860] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 1244.594589] Key type lgssc unregistered [ 1244.733385] LNet: 81751:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1245.797317] LNet: Removed LNI 192.168.203.144@tcp [ 1246.140081] Key type .llcrypt unregistered [ 1246.141916] Key type ._llcrypt unregistered [ 1254.857528] Key type ._llcrypt registered [ 1254.858647] Key type .llcrypt registered [ 1254.890443] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 1255.207466] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1255.250883] alg: No test for adler32 (adler32-zlib) [ 1256.101948] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 1256.181898] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 1256.183872] LNet: Accept secure, port 988 [ 1257.759134] Key type lgssc registered [ 1258.079932] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1260.549286] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1260.557534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1261.701001] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1262.959758] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug +malloc all [ 1265.618814] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1265.709427] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1266.921523] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug +malloc all [ 1268.501276] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1269.684244] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1271.808265] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1271.892616] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1273.656300] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug +malloc all [ 1275.259215] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1286.241757] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000402:35 to 0x280000402:65) [ 1295.839783] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1295.843142] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1295.848439] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1296.863771] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1296.863910] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1296.864338] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1299.500064] LustreError: 85364:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1299.562288] Lustre: server umount lustre-OST0000 complete [ 1301.471621] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1301.474793] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1301.478212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1301.479893] Lustre: Skipped 1 previous similar message [ 1307.103906] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1307.105595] Lustre: Skipped 2 previous similar messages [ 1312.223532] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1312.225870] Lustre: Skipped 1 previous similar message [ 1315.295101] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1315.325581] LustreError: 85565:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1315.327413] LustreError: 85565:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1315.349578] Lustre: server umount lustre-MDT0000 complete [ 1316.496526] LustreError: 83094:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097145 with bad export cookie 4471139908914761171 [ 1316.497752] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1316.499951] LustreError: 83094:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1316.553693] LustreError: 85769:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 1316.555587] LustreError: 85769:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1316.624870] Lustre: server umount lustre-MDT0001 complete [ 1321.217418] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 1322.311551] Key type lgssc unregistered [ 1322.456340] LNet: 86296:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1323.492327] LNet: Removed LNI 192.168.203.144@tcp [ 1323.802317] Key type .llcrypt unregistered [ 1323.803754] Key type ._llcrypt unregistered [ 1332.551684] Key type ._llcrypt registered [ 1332.552729] Key type .llcrypt registered [ 1332.588563] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 1332.903272] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1332.936375] alg: No test for adler32 (adler32-zlib) [ 1333.779165] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 1333.848887] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 1333.850321] LNet: Accept secure, port 988 [ 1335.431189] Key type lgssc registered [ 1335.802163] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1338.277238] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1338.284738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1355.743266] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1361.887214] LustreError: 87631:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.144@tcp: failed processing log, type 4: rc = -110 [ 1391.583161] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1393.588376] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1395.474169] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1397.898964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1397.926157] Lustre: *** cfs_fail_loc=706, val=0*** [ 1398.047891] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1398.068208] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000402:35 to 0x280000402:97) [ 1399.198174] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1401.755892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1401.866670] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1402.996298] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1404.586418] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1405.637435] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1406.944078] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1406.945718] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1406.948598] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1408.363072] LustreError: 89934:0:(obd_class.h:479:obd_check_dev()) Device 4 not setup [ 1408.412654] Lustre: server umount lustre-OST0000 complete [ 1410.016995] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1410.017693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1410.020517] Lustre: Skipped 1 previous similar message [ 1415.136906] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1415.139947] Lustre: Skipped 2 previous similar messages [ 1419.234910] LustreError: 90135:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup [ 1419.237229] LustreError: 90135:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1419.259984] Lustre: server umount lustre-MDT0000 complete [ 1420.255949] LustreError: 88601:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1420.259692] LustreError: 88601:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 3 previous similar messages [ 1420.412103] LustreError: 87640:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097249 with bad export cookie 6308931060018519394 [ 1420.414488] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1420.415311] LustreError: 87640:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1420.478162] LustreError: 90339:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1420.480632] LustreError: 90339:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1420.552034] Lustre: server umount lustre-MDT0001 complete [ 1423.944847] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 1424.912403] Key type lgssc unregistered [ 1425.042308] LNet: 90865:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1426.085298] LNet: Removed LNI 192.168.203.144@tcp [ 1426.369485] Key type .llcrypt unregistered [ 1426.370379] Key type ._llcrypt unregistered [ 1434.577048] Key type ._llcrypt registered [ 1434.578320] Key type .llcrypt registered [ 1434.612132] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 1434.923798] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1434.951393] alg: No test for adler32 (adler32-zlib) [ 1435.809269] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 1435.887890] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 1435.890378] LNet: Accept secure, port 988 [ 1437.471130] Key type lgssc registered [ 1437.791240] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1440.288055] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1440.295145] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1442.423407] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1445.039559] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1445.226698] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1446.660738] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1449.585431] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1449.696436] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1451.718542] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1454.478722] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1454.482925] Lustre: MGS: Client 4d4d5d7c-f77d-45cd-b659-2ec436c3cfe3 (at 0@lo) reconnecting [ 1454.566280] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1454.597840] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000402:35 to 0x280000402:129) [ 1455.859471] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1464.800032] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1464.802380] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1464.806696] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1465.826145] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1465.826638] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1465.827393] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1465.836986] Lustre: Skipped 1 previous similar message [ 1469.612230] LustreError: 94849:0:(obd_class.h:479:obd_check_dev()) Device 14 not setup [ 1469.674448] Lustre: server umount lustre-OST0000 complete [ 1470.944372] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1470.944577] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1470.945019] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1470.951422] Lustre: Skipped 1 previous similar message [ 1476.064703] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1476.067365] Lustre: Skipped 2 previous similar messages [ 1481.184443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1481.187577] Lustre: Skipped 1 previous similar message [ 1485.279098] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1485.312892] LustreError: 95051:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1485.315453] LustreError: 95051:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1485.323782] Lustre: server umount lustre-MDT0000 complete [ 1486.304173] LustreError: 92910:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1486.309737] LustreError: 92910:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 6 previous similar messages [ 1486.544428] LustreError: 95253:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1486.547044] LustreError: 95253:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1486.621105] Lustre: server umount lustre-MDT0001 complete [ 1487.757866] Lustre: server umount MGS complete [ 1491.221861] Lustre: DEBUG MARKER: == conf-sanity test 41b: mount mds with --nosvc and --nomgs on first mount ========================================================== 11:01:59 (1755097319) [ 1500.336407] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 1503.190793] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 1506.339455] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1508.480866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1510.427545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1512.472407] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1514.720790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1514.742801] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1516.078531] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1520.084419] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1520.113827] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1520.215321] Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0001 [ 1520.239549] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1520.278841] Lustre: lustre-MDT0001: new disk, initializing [ 1520.308237] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1520.317374] Lustre: srv-lustre-MDT0001: Waiting to contact MDT0000 to allocate super-sequence: rc = -115 [ 1520.320568] Lustre: 99939:0:(lod_sub_object.c:951:lod_sub_prep_llog()) lustre-MDT0001-mdtlov: can't create new llog: rc = -115 [ 1520.323257] LustreError: 99939:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osd: get update log duration 0, retries 0, failed: rc = -115 [ 1521.744119] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1525.982773] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1526.005701] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1526.098409] Lustre: lustre-OST0000: new disk, initializing [ 1526.100702] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1526.132058] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1528.177137] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1530.911703] Lustre: 101124:0:(client.c:1611:after_reply()) @@@ resending request on EINPROGRESS req@0000000038f35e2e x1840352868730240/t0(0) o700->lustre-OST0000-osc-MDT0001@0@lo:31/4 lens 264/248 e 0 to 0 dl 1755097376 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'' uid:0 gid:0 projid:4294967295 [ 1531.261648] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1533.430770] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1533.436372] Lustre: MGS: Client be78a858-388e-4961-9794-f05d86f1b209 (at 0@lo) reconnecting [ 1533.451750] Lustre: 99229:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0000/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1533.460858] LustreError: 99229:0:(mgs_handler.c:244:mgs_revoke_lock()) MGS: can't take cfg lock for 0x65727473756c/0x0 : rc = -11 [ 1533.472555] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1533.509286] Lustre: lustre-MDT0000: new disk, initializing [ 1533.558549] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1534.623900] Lustre: srv-lustre-MDT0001: Waiting to contact MDT0000 to allocate super-sequence: rc = -115 [ 1534.624032] Lustre: 101627:0:(client.c:1611:after_reply()) @@@ resending request on EINPROGRESS req@00000000adb1bd89 x1840352868735488/t0(0) o700->lustre-OST0000-osc-MDT0000@0@lo:31/4 lens 264/248 e 0 to 0 dl 1755097379 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'' uid:0 gid:0 projid:4294967295 [ 1534.636459] Lustre: 101627:0:(client.c:1611:after_reply()) Skipped 1 previous similar message [ 1534.820569] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1539.682765] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1539.686716] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1539.690227] Lustre: Skipped 1 previous similar message [ 1539.697446] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000400 [ 1560.544034] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1560.544159] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1560.546769] LustreError: Skipped 1 previous similar message [ 1560.547165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1560.550719] Lustre: Skipped 1 previous similar message [ 1572.831215] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1572.854978] LustreError: 102156:0:(obd_class.h:479:obd_check_dev()) Device 11 not setup [ 1572.858154] LustreError: 102156:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1572.928219] Lustre: server umount lustre-OST0000 complete [ 1577.952483] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1577.952822] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1577.955687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1577.960157] Lustre: Skipped 1 previous similar message [ 1577.963081] Lustre: Skipped 7 previous similar messages [ 1580.418925] LustreError: 102360:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 1580.421506] LustreError: 102360:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1580.431359] Lustre: server umount lustre-MDT0000 complete [ 1596.383133] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1596.415800] LustreError: 102562:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 1596.417975] LustreError: 102562:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1596.512448] Lustre: server umount lustre-MDT0001 complete [ 1597.799311] Lustre: server umount MGS complete [ 1601.652069] Lustre: DEBUG MARKER: == conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one ========================================================== 11:03:49 (1755097429) [ 1608.457139] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 1609.582539] Key type lgssc unregistered [ 1609.718463] LNet: 104096:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1610.789380] LNet: Removed LNI 192.168.203.144@tcp [ 1611.084355] Key type .llcrypt unregistered [ 1611.085718] Key type ._llcrypt unregistered [ 1618.037568] Key type ._llcrypt registered [ 1618.038536] Key type .llcrypt registered [ 1618.080896] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 1618.554031] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1618.570215] alg: No test for adler32 (adler32-zlib) [ 1619.433037] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 1619.520513] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 1619.522608] LNet: Accept secure, port 988 [ 1621.111181] Key type lgssc registered [ 1621.482842] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1624.001865] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1624.009708] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1625.168139] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1626.457157] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1629.295501] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1629.432444] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1630.686213] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1632.408318] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1633.605373] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1635.796800] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1635.905319] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1637.722374] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1639.380667] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1650.277276] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:3 to 0x280000400:33) [ 1659.871946] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1659.875138] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1659.880278] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1660.896882] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1660.897723] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1660.897747] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1660.898973] Lustre: Skipped 1 previous similar message [ 1663.467085] LustreError: 107522:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1663.531302] Lustre: server umount lustre-OST0000 complete [ 1665.504032] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1665.507176] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1665.512572] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1671.136755] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1671.139481] Lustre: Skipped 3 previous similar messages [ 1676.256033] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1676.257950] Lustre: Skipped 1 previous similar message [ 1679.327158] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1679.365904] LustreError: 107724:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1679.367980] LustreError: 107724:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1679.398221] Lustre: server umount lustre-MDT0000 complete [ 1680.655357] LustreError: 106879:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097509 with bad export cookie 5057124394195110852 [ 1680.658386] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1680.660146] LustreError: 106879:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1680.728228] LustreError: 107928:0:(obd_class.h:479:obd_check_dev()) Device 15 not setup [ 1680.730175] LustreError: 107928:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1680.812440] Lustre: server umount lustre-MDT0001 complete [ 1685.282642] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 1686.411175] Key type lgssc unregistered [ 1686.541563] LNet: 108454:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1687.589767] LNet: Removed LNI 192.168.203.144@tcp [ 1687.925569] Key type .llcrypt unregistered [ 1687.926556] Key type ._llcrypt unregistered [ 1694.215796] Key type ._llcrypt registered [ 1694.217377] Key type .llcrypt registered [ 1694.256683] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 1694.667825] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1694.678366] alg: No test for adler32 (adler32-zlib) [ 1695.524687] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 1695.603763] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 1695.606586] LNet: Accept secure, port 988 [ 1697.191142] Key type lgssc registered [ 1697.522501] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1699.481839] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1699.481844] LustreError: 109483:0:(tgt_mount.c:2224:server_fill_super()) cfs_race id 716 sleeping [ 1699.484044] Lustre: Skipped 1 previous similar message [ 1699.489026] LustreError: 109484:0:(tgt_mount.c:2224:server_fill_super()) cfs_fail_race id 716 waking [ 1699.491522] LustreError: 109483:0:(tgt_mount.c:2224:server_fill_super()) cfs_fail_race id 716 awake: rc=4996 [ 1699.493718] LustreError: 109483:0:(tgt_mount.c:2236:server_fill_super()) Unable to start osd on /dev/mapper/mds1_flakey: -114 [ 1699.497541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1699.499018] LustreError: 109483:0:(super25.c:181:lustre_fill_super()) llite: Unable to mount : rc = -114 [ 1700.651889] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1702.990133] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1703.075255] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1704.235473] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1705.691305] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing lsmod [ 1706.626927] LustreError: 110647:0:(tgt_mount.c:2224:server_fill_super()) cfs_race id 716 sleeping [ 1706.628690] LustreError: 110646:0:(tgt_mount.c:2224:server_fill_super()) cfs_fail_race id 716 waking [ 1706.631403] LustreError: 110647:0:(tgt_mount.c:2224:server_fill_super()) cfs_fail_race id 716 awake: rc=4998 [ 1706.634860] LustreError: 110647:0:(tgt_mount.c:2236:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -114 [ 1706.637147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1706.638575] LustreError: 110647:0:(super25.c:181:lustre_fill_super()) llite: Unable to mount : rc = -114 [ 1706.751369] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1707.631245] LustreError: 109553:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1707.634682] LustreError: 109553:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1707.637235] LustreError: 109553:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1708.511850] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1708.514405] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1708.519580] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1712.097550] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1712.098042] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1712.099341] Lustre: Skipped 1 previous similar message [ 1712.103966] Lustre: Skipped 1 previous similar message [ 1713.123183] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1713.125404] Lustre: Skipped 3 previous similar messages [ 1713.790228] LustreError: 110830:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1713.818898] Lustre: server umount lustre-MDT0000 complete [ 1715.005911] LustreError: 109500:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097544 with bad export cookie 12649866565950364873 [ 1715.009255] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1715.010460] LustreError: 109500:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1715.143245] LustreError: 109957:0:(client.c:1375:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000007f39b66d x1840353141340416/t0(0) o900->lustre-MDT0000-lwp-MDT0001@0@lo:29/10 lens 264/248 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1715.153395] LustreError: 111037:0:(obd_class.h:479:obd_check_dev()) Device 15 not setup [ 1715.156123] LustreError: 111037:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1715.230477] Lustre: server umount lustre-MDT0001 complete [ 1726.451430] LustreError: 111238:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 1726.454083] LustreError: 111238:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1726.525474] Lustre: server umount lustre-OST0000 complete [ 1729.206070] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1729.364881] LustreError: 111714:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1729.370159] LustreError: 111714:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 1729.387537] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1730.715465] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1733.517590] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1733.632047] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1734.859141] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1736.567703] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1737.689150] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1739.919672] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1741.909303] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1743.790021] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1754.408421] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:3 to 0x280000400:65) [ 1763.807908] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1763.810694] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1763.815438] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1764.836152] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1766.826972] LustreError: 113963:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1766.828827] LustreError: 113963:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1766.890191] Lustre: server umount lustre-OST0000 complete [ 1769.439780] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1769.441662] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1769.444876] Lustre: Skipped 1 previous similar message [ 1769.446863] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1769.448748] Lustre: Skipped 2 previous similar messages [ 1774.205271] LustreError: 114166:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1774.206964] LustreError: 114166:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1774.232260] Lustre: server umount lustre-MDT0000 complete [ 1775.072914] LustreError: 111713:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1775.076316] LustreError: 111713:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 1775.470930] LustreError: 111739:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097604 with bad export cookie 12649866565950365524 [ 1775.473053] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1775.474518] LustreError: 111739:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1775.617760] Lustre: server umount lustre-MDT0001 complete [ 1779.959667] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 1781.011245] Key type lgssc unregistered [ 1781.145283] LNet: 114900:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1782.181427] LNet: Removed LNI 192.168.203.144@tcp [ 1782.489187] Key type .llcrypt unregistered [ 1782.489910] Key type ._llcrypt unregistered [ 1791.276745] Key type ._llcrypt registered [ 1791.277753] Key type .llcrypt registered [ 1791.317110] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 1791.698192] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1791.742056] alg: No test for adler32 (adler32-zlib) [ 1792.594793] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 1792.674959] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 1792.676636] LNet: Accept secure, port 988 [ 1794.255177] Key type lgssc registered [ 1794.661633] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1797.350968] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1797.359073] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1798.531031] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1799.885324] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1802.909839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1803.045817] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1804.433297] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1806.377602] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1807.653936] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1809.970777] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1810.087511] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1811.841939] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1813.583299] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1823.398447] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:67 to 0x280000400:97) [ 1825.821795] Lustre: Setting parameter lustre-client.llite.some_wrong_param=10 in log lustre-client [ 1826.615748] Lustre: Modifying parameter lustre-client.llite.some_wrong_param=20 in log lustre-client [ 1828.833862] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1828.833987] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1828.836196] LustreError: Skipped 1 previous similar message [ 1828.837115] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1828.840901] Lustre: Skipped 1 previous similar message [ 1829.613248] LustreError: 118607:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1829.671907] Lustre: server umount lustre-OST0000 complete [ 1833.019614] LustreError: 118808:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1833.022758] LustreError: 118808:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1833.050774] Lustre: server umount lustre-MDT0000 complete [ 1833.952519] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1833.952931] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1833.953390] LustreError: 116258:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1833.953396] LustreError: 116258:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 1833.967664] Lustre: Skipped 1 previous similar message [ 1834.314292] LustreError: 118461:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097663 with bad export cookie 17365870594576972336 [ 1834.314791] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1834.317508] LustreError: 118461:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1834.384963] LustreError: 119011:0:(obd_class.h:479:obd_check_dev()) Device 15 not setup [ 1834.387503] LustreError: 119011:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1834.469190] Lustre: server umount lustre-MDT0001 complete [ 1839.353475] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 1840.528529] Key type lgssc unregistered [ 1840.681407] LNet: 119537:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1841.700446] LNet: Removed LNI 192.168.203.144@tcp [ 1842.047883] Key type .llcrypt unregistered [ 1842.049549] Key type ._llcrypt unregistered [ 1849.239062] Key type ._llcrypt registered [ 1849.240229] Key type .llcrypt registered [ 1849.275280] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 1849.620517] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1849.634133] alg: No test for adler32 (adler32-zlib) [ 1850.494406] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 1850.577686] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 1850.580244] LNet: Accept secure, port 988 [ 1852.159112] Key type lgssc registered [ 1852.515566] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1855.156135] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1855.165151] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1856.326543] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1857.702467] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1860.662784] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1860.804024] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1862.249464] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1864.030955] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1865.138748] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1867.282131] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1867.388834] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1869.083925] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1870.679876] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1881.763259] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 1891.295771] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1891.298711] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1891.303761] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1892.322017] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1892.322305] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1892.322312] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1894.956067] LustreError: 122966:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1895.016881] Lustre: server umount lustre-OST0000 complete [ 1896.928042] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1896.931705] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1896.936607] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1896.938088] Lustre: Skipped 1 previous similar message [ 1902.330289] LustreError: 123168:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1902.332235] LustreError: 123168:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1902.356141] Lustre: server umount lustre-MDT0000 complete [ 1902.561175] LustreError: 120735:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1902.564731] LustreError: 120735:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 3 previous similar messages [ 1903.509063] LustreError: 122322:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097732 with bad export cookie 15893408419677111355 [ 1903.512325] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1903.513946] LustreError: 122322:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1903.577243] LustreError: 123369:0:(obd_class.h:479:obd_check_dev()) Device 15 not setup [ 1903.580780] LustreError: 123369:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1903.661715] Lustre: server umount lustre-MDT0001 complete [ 1908.052566] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 1909.113573] Key type lgssc unregistered [ 1909.255384] LNet: 123895:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1910.309270] LNet: Removed LNI 192.168.203.144@tcp [ 1910.599190] Key type .llcrypt unregistered [ 1910.600574] Key type ._llcrypt unregistered [ 1918.882734] Key type ._llcrypt registered [ 1918.883718] Key type .llcrypt registered [ 1918.918255] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 1919.233920] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1919.265116] alg: No test for adler32 (adler32-zlib) [ 1920.109317] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 1920.180332] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 1920.181893] LNet: Accept secure, port 988 [ 1921.759156] Key type lgssc registered [ 1922.064862] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1924.401125] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1924.406452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1925.535095] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1926.859514] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1929.783994] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1929.911391] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1931.128681] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1932.860245] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1934.080362] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1936.403402] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1936.527296] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1938.524352] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 1940.374736] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1950.884095] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 1958.472504] Lustre: Setting parameter lustre-MDT0000.mdt.root_squash=0:0 in log lustre-MDT0000 [ 1959.777790] Lustre: Setting parameter lustre-MDT0000.mdt.nosquash_nids=NONE in log lustre-MDT0000 [ 1959.779897] Lustre: Skipped 2 previous similar messages [ 1961.180967] Lustre: Modifying parameter lustre-MDT0000.mdt.root_squash=500:500 in log lustre-MDT0000 [ 1961.184452] Lustre: Skipped 2 previous similar messages [ 1965.047708] Lustre: lustre-MDT0001: nosquash_nids is cleared [ 1965.057708] Lustre: lustre-MDT0001: root_squash is set to 500:500 [ 1967.300814] Lustre: Modifying parameter lustre-MDT0000.mdt.nosquash_nids=2@gni 0@lo 192.168.203.44@tcp 192.168.0.[2,10]@tcp in log lustre-MDT0000 [ 1967.303467] Lustre: Skipped 2 previous similar messages [ 1971.190451] Lustre: lustre-MDT0001: nosquash_nids set to 2@gni 0@lo 192.168.203.44@tcp 192.168.0.[2,10]@tcp [ 1974.528213] Lustre: Modifying parameter lustre-MDT0000.mdt.nosquash_nids=192.168.203.44/24@tcp in log lustre-MDT0000 [ 1974.530395] Lustre: Skipped 2 previous similar messages [ 1981.434870] Lustre: lustre-MDT0001: nosquash_nids set to 192.168.203.44/24@tcp [ 1981.436679] Lustre: Skipped 1 previous similar message [ 1985.579067] LustreError: 129104:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1985.635352] Lustre: server umount lustre-OST0000 complete [ 1987.040796] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1987.041291] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1987.046016] Lustre: Skipped 1 previous similar message [ 1987.049242] Lustre: Skipped 1 previous similar message [ 1992.160314] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1992.161774] Lustre: Skipped 1 previous similar message [ 1996.258538] LustreError: 129305:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1996.260870] LustreError: 129305:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1996.286799] Lustre: server umount lustre-MDT0000 complete [ 1997.280545] LustreError: 125260:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1997.284641] LustreError: 125260:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 3 previous similar messages [ 1997.521127] LustreError: 125242:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097826 with bad export cookie 2300381276233726772 [ 1997.524022] LustreError: 125242:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1997.524920] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2007.010493] LustreError: 129507:0:(obd_class.h:479:obd_check_dev()) Device 15 not setup [ 2007.012095] LustreError: 129507:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2007.083465] Lustre: server umount lustre-MDT0001 complete [ 2011.743817] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 2012.837908] Key type lgssc unregistered [ 2012.992485] LNet: 130034:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2014.053376] LNet: Removed LNI 192.168.203.144@tcp [ 2014.405575] Key type .llcrypt unregistered [ 2014.407444] Key type ._llcrypt unregistered [ 2024.735907] Key type ._llcrypt registered [ 2024.736978] Key type .llcrypt registered [ 2024.775152] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 2025.143788] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2025.202318] alg: No test for adler32 (adler32-zlib) [ 2026.063472] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 2026.154575] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 2026.156242] LNet: Accept secure, port 988 [ 2027.743266] Key type lgssc registered [ 2028.113960] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2030.886363] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2030.895805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2032.066568] Lustre: lustre-MDT0000: root_squash is set to 500:500 [ 2032.071604] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2032.077813] Lustre: lustre-MDT0000: nosquash_nids set to 192.168.203.44/24@tcp [ 2033.418591] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2036.343496] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2036.455635] Lustre: lustre-MDT0001: root_squash is set to 500:500 [ 2036.463699] Lustre: lustre-MDT0001: nosquash_nids set to 192.168.203.44/24@tcp [ 2036.479698] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2037.786571] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2039.607512] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2040.903833] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2043.270208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2043.404962] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2045.317803] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2047.149362] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2057.767544] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:169 to 0x280000400:193) [ 2062.819720] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2062.819961] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2062.821457] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2062.828490] Lustre: Skipped 1 previous similar message [ 2063.917270] LustreError: 133847:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2063.990487] Lustre: server umount lustre-OST0000 complete [ 2067.327690] LustreError: 134047:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2067.329425] LustreError: 134047:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2067.359487] Lustre: server umount lustre-MDT0000 complete [ 2067.423681] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2067.426228] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2067.430045] LustreError: 131542:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2067.433656] LustreError: 131542:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 2067.937603] LustreError: 131568:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2068.592088] LustreError: 131522:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097897 with bad export cookie 3555909691943797003 [ 2068.594984] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2068.595375] LustreError: 131522:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2068.669351] LustreError: 134248:0:(obd_class.h:479:obd_check_dev()) Device 15 not setup [ 2068.671141] LustreError: 134248:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2068.754442] Lustre: server umount lustre-MDT0001 complete [ 2073.466141] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 2074.708910] Key type lgssc unregistered [ 2074.856460] LNet: 134774:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2075.877497] LNet: Removed LNI 192.168.203.144@tcp [ 2076.213472] Key type .llcrypt unregistered [ 2076.214903] Key type ._llcrypt unregistered [ 2085.262955] Key type ._llcrypt registered [ 2085.264033] Key type .llcrypt registered [ 2085.306618] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 2091.457400] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 2091.805711] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2091.853382] alg: No test for adler32 (adler32-zlib) [ 2092.718615] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 2092.807722] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 2092.810493] LNet: Accept secure, port 988 [ 2094.391176] Key type lgssc registered [ 2094.766413] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2097.806173] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2099.962455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2101.887440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2103.871226] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2106.113264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2106.133589] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2106.141180] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2107.230763] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2107.242706] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2107.279643] Lustre: lustre-MDT0000: new disk, initializing [ 2107.303707] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2107.313252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2108.692776] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2112.717644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2112.739092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2112.768300] Lustre: 138117:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2112.789200] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2112.790848] Lustre: Skipped 1 previous similar message [ 2112.828955] Lustre: lustre-MDT0001: new disk, initializing [ 2112.862576] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2112.874187] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2112.877193] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2114.176906] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2117.176371] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2118.317022] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2120.505678] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2120.530266] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2120.628724] Lustre: lustre-OST0000: new disk, initializing [ 2120.630573] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2120.659625] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2121.849976] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2121.855580] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2121.886224] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2122.524070] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2125.512206] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2126.835581] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2126.929793] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2128.161461] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2128.239318] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2132.449260] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2132.451038] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2132.452077] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2132.454554] Lustre: Skipped 1 previous similar message [ 2132.457769] Lustre: Skipped 1 previous similar message [ 2137.570425] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2137.572048] Lustre: Skipped 1 previous similar message [ 2142.690130] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2142.692104] Lustre: Skipped 1 previous similar message [ 2143.199132] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2143.215414] LustreError: 141025:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2143.281905] Lustre: server umount lustre-OST0000 complete [ 2147.808650] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2147.809557] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2147.813969] Lustre: Skipped 1 previous similar message [ 2150.656912] LustreError: 141227:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2150.659401] LustreError: 141227:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2150.685064] Lustre: server umount lustre-MDT0000 complete [ 2151.857121] LustreError: 138890:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755097980 with bad export cookie 16971316659584444149 [ 2151.860189] LustreError: 138890:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2151.860460] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2151.918809] LustreError: 141429:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2151.920365] LustreError: 141429:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2152.002445] Lustre: server umount lustre-MDT0001 complete [ 2154.888458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2155.070078] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2156.249928] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2159.011041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2159.123564] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2160.430732] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2162.129132] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2163.335275] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2165.514366] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2167.450471] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2169.143562] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2170.547586] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2180.853413] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 10 sec [ 2182.162280] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2182.240481] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2185.675640] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2185.700501] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2185.758741] Lustre: lustre-OST0001: new disk, initializing [ 2185.760986] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2185.788243] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2185.790655] Lustre: Skipped 1 previous similar message [ 2187.666977] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2190.841132] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2191.359983] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2191.363195] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2191.397521] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2191.936954] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 2193.246972] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2193.324310] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2194.499312] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9fe960bf6800.ost_server_uuid 50 [ 2194.992028] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9fe960bf6800.ost_server_uuid in FULL state after 0 sec [ 2201.570714] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2201.572574] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2201.574463] LustreError: Skipped 2 previous similar messages [ 2201.581575] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2201.584533] Lustre: Skipped 2 previous similar messages [ 2202.860505] LustreError: 146176:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 2202.862142] LustreError: 146176:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2202.878240] Lustre: server umount lustre-OST0001 complete [ 2205.664311] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2205.664331] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2205.666194] LustreError: Skipped 1 previous similar message [ 2205.669549] Lustre: Skipped 1 previous similar message [ 2206.691107] LustreError: 146177:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2206.695474] LustreError: 146177:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 3 previous similar messages [ 2210.222097] LustreError: 146378:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2210.224039] LustreError: 146378:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2210.287946] Lustre: server umount lustre-OST0000 complete [ 2211.808454] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2211.809318] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2211.811823] Lustre: Skipped 2 previous similar messages [ 2211.815349] Lustre: Skipped 5 previous similar messages [ 2217.686828] Lustre: server umount lustre-MDT0000 complete [ 2218.879047] LustreError: 143509:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755098047 with bad export cookie 16971316659584445017 [ 2218.881547] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2218.881906] LustreError: 143509:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2218.949052] LustreError: 146781:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 2218.951144] LustreError: 146781:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 2219.038390] Lustre: server umount lustre-MDT0001 complete [ 2226.465203] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 2227.584758] Key type lgssc unregistered [ 2227.733396] LNet: 147763:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2228.773433] LNet: Removed LNI 192.168.203.144@tcp [ 2229.094915] Key type .llcrypt unregistered [ 2229.095975] Key type ._llcrypt unregistered [ 2239.781553] Key type ._llcrypt registered [ 2239.782585] Key type .llcrypt registered [ 2239.821772] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 2246.052221] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 2246.524401] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2246.591925] alg: No test for adler32 (adler32-zlib) [ 2247.442575] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 2247.534517] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 2247.536186] LNet: Accept secure, port 988 [ 2249.119145] Key type lgssc registered [ 2249.445786] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2252.419967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2254.662142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2256.556055] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2258.697977] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2261.025472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2261.046567] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2261.051921] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2262.149410] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2262.160700] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2262.202272] Lustre: lustre-MDT0000: new disk, initializing [ 2262.228769] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2262.234960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2263.561809] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2267.768285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2267.787988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2267.818719] Lustre: 151669:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2267.842379] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2267.844080] Lustre: Skipped 1 previous similar message [ 2267.878577] Lustre: lustre-MDT0001: new disk, initializing [ 2267.909383] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2267.921536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2267.924970] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2269.353359] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2272.567530] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2273.913159] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2276.364566] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2276.392033] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2276.497912] Lustre: lustre-OST0000: new disk, initializing [ 2276.499581] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2276.526845] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2278.450071] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2279.678527] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2279.681990] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2279.723996] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2281.623604] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2283.014287] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2283.089482] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2284.390277] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2284.472227] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2287.149091] LustreError: 154575:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2287.213366] Lustre: server umount lustre-OST0000 complete [ 2288.608074] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2288.612507] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2288.620179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2290.145229] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2290.146297] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2295.264231] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2295.266194] Lustre: Skipped 2 previous similar messages [ 2298.339193] LustreError: 154775:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2298.341214] LustreError: 154775:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2298.368622] Lustre: server umount lustre-MDT0000 complete [ 2299.650075] LustreError: 151659:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755098128 with bad export cookie 163482461876270070 [ 2299.652995] LustreError: 151659:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2299.653096] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2309.090864] LustreError: 154977:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2309.092781] LustreError: 154977:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2309.178954] Lustre: server umount lustre-MDT0001 complete [ 2311.822954] Lustre: DEBUG MARKER: == conf-sanity test 47a: server restart does not lose lru_resize ========================================================== 11:15:39 (1755098139) [ 2316.896747] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 2320.012244] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 2323.310229] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2325.454561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2327.389693] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2329.483103] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2331.816119] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2331.837436] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2331.919585] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2331.933391] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2331.972839] Lustre: lustre-MDT0000: new disk, initializing [ 2331.997635] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2332.003879] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2333.282321] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2337.398533] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2337.420767] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2337.458139] Lustre: 157946:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2337.481882] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2337.483620] Lustre: Skipped 1 previous similar message [ 2337.519455] Lustre: lustre-MDT0001: new disk, initializing [ 2337.549335] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2337.560206] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2337.563138] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2338.832352] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2341.931222] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2343.221478] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2345.545364] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2345.570221] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2345.666224] Lustre: lustre-OST0000: new disk, initializing [ 2345.668139] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2345.696444] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2347.597155] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2350.735362] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2354.167934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2354.172523] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2354.199599] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2358.160299] Lustre: Failing over lustre-OST0000 [ 2358.195099] LustreError: 160372:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2358.196825] LustreError: 160372:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2358.277276] Lustre: server umount lustre-OST0000 complete [ 2371.148075] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2371.253660] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2371.259674] Lustre: Skipped 1 previous similar message [ 2371.264470] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2371.271934] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 2372.725745] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2373.129337] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2374.377251] Lustre: Failing over lustre-MDT0000 [ 2374.447372] LustreError: 161337:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2374.449881] LustreError: 161337:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2374.480658] Lustre: server umount lustre-MDT0000 complete [ 2375.391106] Lustre: 149893:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755098188/real 1755098188] req@000000004705b762 x1840353720196736/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1755098204 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2375.401449] Lustre: 149893:0:(client.c:2453:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 2376.673470] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2376.677175] Lustre: Skipped 2 previous similar messages [ 2380.767109] Lustre: 149893:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755098193/real 1755098193] req@00000000fe8b12e1 x1840353720198656/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1755098209 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2381.792177] LustreError: 158727:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2381.800280] LustreError: 158727:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 7 previous similar messages [ 2382.956532] LustreError: 160272:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.44@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2385.823089] Lustre: 149892:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755098198/real 1755098198] req@00000000999d222f x1840353720199808/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1755098214 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2386.911986] LustreError: 158727:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2386.916699] LustreError: 158727:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 2387.433769] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2387.469866] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2387.473335] Lustre: 157949:0:(mgc_request.c:1854:mgc_process_log()) MGC192.168.203.144@tcp: IR log lustre-mdtir failed, not fatal: rc = -5 [ 2387.543061] LustreError: lustre-OST0000-osc-MDT0000: operation ost_connect to node 0@lo failed: rc = -114 [ 2387.581473] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2388.076458] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 2388.759634] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2393.061546] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to (at 0@lo) [ 2393.068645] Lustre: lustre-OST0000: Recovery over after 0:21, of 3 clients 3 recovered and 0 were evicted. [ 2393.071080] Lustre: Skipped 1 previous similar message [ 2393.093014] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2398.178527] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2398.180469] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2398.180555] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2398.182020] Lustre: Skipped 1 previous similar message [ 2400.171102] LustreError: 162225:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2400.173208] LustreError: 162225:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2400.232321] Lustre: server umount lustre-OST0000 complete [ 2403.297222] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2403.297815] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2403.300827] LustreError: Skipped 1 previous similar message [ 2403.305061] Lustre: Skipped 2 previous similar messages [ 2407.571426] Lustre: server umount lustre-MDT0000 complete [ 2408.415981] LustreError: 157957:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2408.420729] LustreError: 157957:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 2408.780844] LustreError: 157938:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755098237 with bad export cookie 163482461876272135 [ 2408.783299] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2408.783608] LustreError: 157938:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2408.927779] Lustre: server umount lustre-MDT0001 complete [ 2413.582353] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 2414.621482] Key type lgssc unregistered [ 2414.754324] LNet: 163154:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2415.781336] LNet: Removed LNI 192.168.203.144@tcp [ 2416.074487] Key type .llcrypt unregistered [ 2416.075519] Key type ._llcrypt unregistered [ 2436.131724] Key type ._llcrypt registered [ 2436.132597] Key type .llcrypt registered [ 2436.175334] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 2436.640590] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2436.659498] alg: No test for adler32 (adler32-zlib) [ 2437.520847] Lustre: Lustre: Build Version: 2.16.56_9_g5b8a063 [ 2437.609992] LNet: Added LNI 192.168.203.144@tcp [8/256/0/180] [ 2437.611480] LNet: Accept secure, port 988 [ 2439.207188] Key type lgssc registered [ 2439.589794] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2442.331085] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2442.337917] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2443.488494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2444.727189] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2447.806506] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2449.818339] LustreError: 164671:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2449.821522] LustreError: 164671:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2449.823992] LustreError: 164671:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -5 [ 2449.895786] LustreError: 165188:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2449.988598] Lustre: server umount lustre-MDT0000 complete [ 2454.997529] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 2458.280569] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 2461.908721] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2464.221332] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2466.251314] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2468.333104] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2470.648965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2470.674287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2470.762941] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2470.776339] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2470.811847] Lustre: lustre-MDT0000: new disk, initializing [ 2470.838137] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2470.847149] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2472.104780] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2476.289086] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2476.314750] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2476.346367] Lustre: 167811:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2476.374943] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2476.376332] Lustre: Skipped 1 previous similar message [ 2476.412157] Lustre: lustre-MDT0001: new disk, initializing [ 2476.445323] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2476.457048] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2476.460020] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2477.767294] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2480.889107] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2482.098657] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2484.453261] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2484.477721] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2484.570435] Lustre: lustre-OST0000: new disk, initializing [ 2484.572111] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2484.598654] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2486.272629] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2486.277224] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2486.321696] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2486.511259] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2489.651765] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2491.546626] Lustre: DEBUG MARKER: SKIP: conf-sanity test_47b Can not create symlink /usr/sbin/lctl [ 2492.099224] Lustre: DEBUG MARKER: == conf-sanity test 48: too many acls on file ============ 11:18:40 (1755098320) [ 2557.919621] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2557.919902] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2557.922827] Lustre: Skipped 2 previous similar messages [ 2557.928241] Lustre: Skipped 2 previous similar messages [ 2561.530371] LustreError: 171291:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2561.532089] LustreError: 171291:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2561.585726] Lustre: server umount lustre-MDT0000 complete [ 2562.820820] LustreError: 168585:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755098391 with bad export cookie 1343260464941978881 [ 2562.822063] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2562.823933] LustreError: 168585:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2563.039473] LustreError: 171190:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2563.039570] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2563.043154] LustreError: 171190:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 2563.047030] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 2567.071444] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 2567.071444] LustreError: 167816:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2572.191484] LustreError: 171176:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2572.191497] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 2577.375112] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2577.375363] LustreError: 171176:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2577.375448] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 2577.403519] LustreError: 171495:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2577.405219] LustreError: 171495:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2577.469817] Lustre: server umount lustre-MDT0001 complete [ 2584.743840] LustreError: 171698:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 2584.747218] LustreError: 171698:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2584.809847] Lustre: server umount lustre-OST0000 complete [ 2587.886533] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 2591.092849] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 2594.506908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2596.743964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2598.768524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2600.769114] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2602.986893] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2603.012864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2603.088039] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2603.099250] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2603.140201] Lustre: lustre-MDT0000: new disk, initializing [ 2603.162476] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2603.167869] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2604.565889] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2608.791221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2608.817499] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2608.847971] Lustre: 174018:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2608.872327] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2608.874112] Lustre: Skipped 1 previous similar message [ 2608.912212] Lustre: lustre-MDT0001: new disk, initializing [ 2608.956924] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2608.959612] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2610.261358] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2613.401652] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2614.683422] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2617.133586] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2617.159490] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2617.259186] Lustre: lustre-OST0000: new disk, initializing [ 2617.261093] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2617.292071] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2617.293967] Lustre: Skipped 1 previous similar message [ 2619.404278] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2622.879536] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2622.967948] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2622.971170] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2623.011962] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2624.376310] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2624.457839] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2625.817615] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2625.896816] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2628.064674] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2628.065291] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2628.066573] LustreError: Skipped 1 previous similar message [ 2628.067910] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2628.069360] Lustre: Skipped 1 previous similar message [ 2632.621085] LustreError: 176924:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2632.622767] LustreError: 176924:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2632.683257] Lustre: server umount lustre-OST0000 complete [ 2634.720144] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2634.723248] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2640.091731] Lustre: server umount lustre-MDT0000 complete [ 2641.413149] LustreError: 174010:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755098470 with bad export cookie 1343260464942137697 [ 2641.415519] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2641.417345] LustreError: 174010:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2641.483576] LustreError: 177330:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2641.485378] LustreError: 177330:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 2641.578750] Lustre: server umount lustre-MDT0001 complete [ 2644.304493] Lustre: DEBUG MARKER: == conf-sanity test 49a: check PARAM_SYS_LDLM_TIMEOUT option of mkfs.lustre ========================================================== 11:21:12 (1755098472) [ 2649.362885] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 2652.445052] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 2655.562149] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2657.740780] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2659.642095] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2661.619353] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2663.774080] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2663.794564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2663.877167] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2663.887702] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2663.933480] Lustre: lustre-MDT0000: new disk, initializing [ 2663.959391] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2663.965403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2665.200058] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2669.249285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2669.274295] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2669.312287] Lustre: 180297:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2669.422832] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2670.707654] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2673.736821] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2674.953614] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2677.106140] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2677.126258] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2679.046118] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2682.035483] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2683.381449] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2683.414096] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2693.602412] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2693.603100] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2693.604655] LustreError: Skipped 1 previous similar message [ 2693.608099] Lustre: Skipped 1 previous similar message [ 2693.608648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2693.612448] Lustre: Skipped 5 previous similar messages [ 2695.340776] LustreError: 182828:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2695.342531] LustreError: 182828:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2695.421569] Lustre: server umount lustre-OST0000 complete [ 2702.811770] Lustre: server umount lustre-MDT0000 complete [ 2703.840023] LustreError: 180308:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2703.846234] LustreError: 180308:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 2704.084201] LustreError: 182040:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755098533 with bad export cookie 1343260464942138565 [ 2704.089116] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2704.245674] Lustre: server umount lustre-MDT0001 complete [ 2707.106885] Lustre: DEBUG MARKER: == conf-sanity test 49b: check PARAM_SYS_LDLM_TIMEOUT option of mkfs.lustre ========================================================== 11:22:15 (1755098535) [ 2712.344223] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_hostid [ 2715.492248] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing load_modules_local [ 2719.051360] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2721.165840] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2723.095588] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2725.117286] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2727.366601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2727.390178] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2727.473347] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2727.489201] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2727.490906] Lustre: Skipped 3 previous similar messages [ 2727.536857] Lustre: lustre-MDT0000: new disk, initializing [ 2727.539275] Lustre: Skipped 2 previous similar messages [ 2727.564722] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2727.567571] Lustre: Skipped 2 previous similar messages [ 2727.574658] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2727.578521] Lustre: Skipped 2 previous similar messages [ 2728.902539] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2732.986278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2733.008690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2733.042194] Lustre: 186200:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2733.156585] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2734.381212] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2737.425326] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2738.602427] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2740.827358] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2740.854895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2741.529880] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2742.778736] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing set_default_debug -1 all [ 2745.908781] Lustre: DEBUG MARKER: oleg344-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2751.971401] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2751.971531] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2751.975554] LustreError: Skipped 1 previous similar message [ 2751.978299] Lustre: Skipped 3 previous similar messages [ 2751.979635] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2751.982919] Lustre: Skipped 3 previous similar messages [ 2753.900120] LustreError: 188724:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2753.903034] LustreError: 188724:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2753.963358] Lustre: server umount lustre-OST0000 complete [ 2761.304894] Lustre: server umount lustre-MDT0000 complete [ 2762.208957] LustreError: 186983:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2762.213880] LustreError: 186983:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 2762.506940] LustreError: 186977:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755098591 with bad export cookie 1343260464942139636 [ 2762.510022] LustreError: 186977:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 2762.510555] LustreError: MGC192.168.203.144@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2762.658233] Lustre: server umount lustre-MDT0001 complete [ 2767.085860] Lustre: DEBUG MARKER: oleg344-server.virtnet: executing unload_modules_local [ 2768.207840] Key type lgssc unregistered [ 2768.363705] LNet: 189654:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2769.381426] LNet: Removed LNI 192.168.203.144@tcp [ 2769.705028] Key type .llcrypt unregistered [ 2769.706107] Key type ._llcrypt unregistered