[ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup subsys cpuacct [ 0.000000] Linux version 3.10.0-7.9-debug (green@centos7-base) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) ) #1 SMP Sat Mar 26 23:28:42 EDT 2022 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:centos7:ext4:ro:-p,-b4096 ro crashkernel=128M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000013edfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000000] e820: last_pfn = 0x13ee00 max_arch_pfn = 0x400000000 [ 0.000000] MTRR default type: write-back [ 0.000000] MTRR fixed ranges enabled: [ 0.000000] 00000-9FFFF write-back [ 0.000000] A0000-BFFFF uncachable [ 0.000000] C0000-FFFFF write-protect [ 0.000000] MTRR variable ranges enabled: [ 0.000000] 0 base 0000C0000000 mask 3FFFC0000000 uncachable [ 0.000000] 1 disabled [ 0.000000] 2 disabled [ 0.000000] 3 disabled [ 0.000000] 4 disabled [ 0.000000] 5 disabled [ 0.000000] 6 disabled [ 0.000000] 7 disabled [ 0.000000] PAT configuration [0-7]: WB WC UC- UC WB WP UC- UC [ 0.000000] e820: last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] mapped at [ffffffffff200b30] [ 0.000000] Base memory trampoline at [ffff880000099000] 99000 size 24576 [ 0.000000] Using GB pages for direct mapping [ 0.000000] BRK [0x023c1000, 0x023c1fff] PGTABLE [ 0.000000] BRK [0x023c2000, 0x023c2fff] PGTABLE [ 0.000000] BRK [0x023c3000, 0x023c3fff] PGTABLE [ 0.000000] BRK [0x023c4000, 0x023c4fff] PGTABLE [ 0.000000] BRK [0x023c5000, 0x023c5fff] PGTABLE [ 0.000000] BRK [0x023c6000, 0x023c6fff] PGTABLE [ 0.000000] RAMDISK: [mem 0xbc2e2000-0xbffbffff] [ 0.000000] Early table checksum verification disabled [ 0.000000] ACPI: RSDP 00000000000f5950 00014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 00000000bffe1bb7 00034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 00000000bffe1a53 00074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 00000000bffe0040 01A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 00000000bffe0000 00040 [ 0.000000] ACPI: APIC 00000000bffe1ac7 00090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 00000000bffe1b57 00038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 00000000bffe1b8f 00028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000013edfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x13e5e3000-0x13e609fff] [ 0.000000] Reserving 128MB of memory at 768MB for crashkernel (System RAM: 4077MB) [ 0.000000] kvm-clock: cpu 0, msr 1:3e592001, primary cpu clock [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 340746026 cycles [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x00001000-0x00ffffff] [ 0.000000] DMA32 [mem 0x01000000-0xffffffff] [ 0.000000] Normal [mem 0x100000000-0x13edfffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x00001000-0x0009efff] [ 0.000000] node 0: [mem 0x00100000-0xbffcdfff] [ 0.000000] node 0: [mem 0x100000000-0x13edfffff] [ 0.000000] Initmem setup node 0 [mem 0x00001000-0x13edfffff] [ 0.000000] On node 0 totalpages: 1043820 [ 0.000000] DMA zone: 64 pages used for memmap [ 0.000000] DMA zone: 21 pages reserved [ 0.000000] DMA zone: 3998 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 12224 pages used for memmap [ 0.000000] DMA32 zone: 782286 pages, LIFO batch:31 [ 0.000000] Normal zone: 4024 pages used for memmap [ 0.000000] Normal zone: 257536 pages, LIFO batch:31 [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] ACPI: IRQ0 used by override. [ 0.000000] ACPI: IRQ5 used by override. [ 0.000000] ACPI: IRQ9 used by override. [ 0.000000] ACPI: IRQ10 used by override. [ 0.000000] ACPI: IRQ11 used by override. [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] e820: [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 38 pages/cpu s115176 r8192 d32280 u524288 [ 0.000000] pcpu-alloc: s115176 r8192 d32280 u524288 alloc=1*2097152 [ 0.000000] pcpu-alloc: [0] 0 1 2 3 [ 0.000000] KVM setup async PF for cpu 0 [ 0.000000] kvm-stealtime: cpu 0, msr 13e2135c0 [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes) [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 1027487 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:centos7:ext4:ro:-p,-b4096 ro crashkernel=128M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] audit: disabled (until reboot) [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] x86/fpu: xstate_offset[2]: 0240, xstate_sizes[2]: 0100 [ 0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form [ 0.000000] Memory: 3820268k/5224448k available (8172k kernel code, 1049168k absent, 355012k reserved, 5773k data, 2532k init) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=4. [ 0.000000] Offload RCU callbacks from all CPUs [ 0.000000] Offload RCU callbacks from CPUs: 0-3. [ 0.000000] NR_IRQS:327936 nr_irqs:456 0 [ 0.000000] Console: colour *CGA 80x25 [ 0.000000] console [ttyS1] enabled [ 0.000000] allocated 25165824 bytes of page_cgroup [ 0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ODEBUG: 73 of 73 active objects replaced [ 0.000000] hpet clockevent registered [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.546164] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.549037] pid_max: default: 32768 minimum: 301 [ 0.550743] Security Framework initialized [ 0.552125] SELinux: Initializing. [ 0.553283] SELinux: Starting in permissive mode [ 0.554972] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) [ 0.559669] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) [ 0.563004] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes) [ 0.565534] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes) [ 0.568612] Initializing cgroup subsys memory [ 0.570225] Initializing cgroup subsys devices [ 0.571849] Initializing cgroup subsys freezer [ 0.573281] Initializing cgroup subsys net_cls [ 0.575038] Initializing cgroup subsys blkio [ 0.576060] Initializing cgroup subsys perf_event [ 0.577179] Initializing cgroup subsys hugetlb [ 0.578426] Initializing cgroup subsys pids [ 0.579275] Initializing cgroup subsys net_prio [ 0.580449] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.582900] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.584269] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.585551] tlb_flushall_shift: 6 [ 0.586337] FEATURE SPEC_CTRL Present [ 0.587462] FEATURE IBPB_SUPPORT Present [ 0.588212] Spectre V2 : Enabling Indirect Branch Prediction Barrier [ 0.590169] Spectre V2 : Vulnerable [ 0.591065] Speculative Store Bypass: Vulnerable [ 0.593770] debug: unmapping init [mem 0xffffffff82019000-0xffffffff8201ffff] [ 0.601514] ACPI: Core revision 20130517 [ 0.604230] ACPI: All ACPI Tables successfully acquired [ 0.605558] ftrace: allocating 30294 entries in 119 pages [ 0.667136] Enabling x2apic [ 0.668230] Enabled x2apic [ 0.669715] Switched APIC routing to physical x2apic. [ 0.673616] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.675872] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (fam: 06, model: 3e, stepping: 04) [ 0.679119] TSC deadline timer enabled [ 0.679356] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.682534] ... version: 2 [ 0.683963] ... bit width: 48 [ 0.685573] ... generic registers: 4 [ 0.687196] ... value mask: 0000ffffffffffff [ 0.688412] ... max period: 00007fffffffffff [ 0.690243] ... fixed-purpose events: 3 [ 0.691776] ... event mask: 000000070000000f [ 0.693887] KVM setup paravirtual spinlock [ 0.700109] kvm-clock: cpu 1, msr 1:3e592041, secondary cpu clock [ 0.703492] KVM setup async PF for cpu 1 [ 0.704351] kvm-stealtime: cpu 1, msr 13e2935c0 [ 0.708159] kvm-clock: cpu 2, msr 1:3e592081, secondary cpu clock [ 0.712875] KVM setup async PF for cpu 2 [ 0.697969] smpboot: Booting Node 0, Processors #1 #2 #3 OK [ 0.713592] kvm-clock: cpu 3, msr 1:3e5920c1, secondary cpu clock [ 0.716419] kvm-stealtime: cpu 2, msr 13e3135c0 [ 0.719358] Brought up 4 CPUs [ 0.719373] KVM setup async PF for cpu 3 [ 0.719382] kvm-stealtime: cpu 3, msr 13e3935c0 [ 0.723715] smpboot: Max logical packages: 1 [ 0.725153] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.730063] devtmpfs: initialized [ 0.731660] x86/mm: Memory block size: 128MB [ 0.737282] EVM: security.selinux [ 0.738309] EVM: security.ima [ 0.739322] EVM: security.capability [ 0.742834] atomic64 test passed for x86-64 platform with CX8 and with SSE [ 0.745780] NET: Registered protocol family 16 [ 0.747037] cpuidle: using governor haltpoll [ 0.748494] ACPI: bus type PCI registered [ 0.749645] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.752505] PCI: Using configuration type 1 for base access [ 0.754711] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.772684] ACPI: Added _OSI(Module Device) [ 0.775564] ACPI: Added _OSI(Processor Device) [ 0.777227] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.779209] ACPI: Added _OSI(Processor Aggregator Device) [ 0.781379] ACPI: Added _OSI(Linux-Dell-Video) [ 0.784291] ACPI: EC: Look up EC in DSDT [ 0.788833] ACPI: Interpreter enabled [ 0.789900] ACPI: (supports S0 S3 S4 S5) [ 0.791271] ACPI: Using IOAPIC for interrupt routing [ 0.793166] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.795744] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.803418] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.805470] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI] [ 0.808221] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM [ 0.810177] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.815168] acpiphp: Slot [2] registered [ 0.816837] acpiphp: Slot [3] registered [ 0.818589] acpiphp: Slot [4] registered [ 0.820454] acpiphp: Slot [5] registered [ 0.822274] acpiphp: Slot [6] registered [ 0.824023] acpiphp: Slot [7] registered [ 0.825808] acpiphp: Slot [8] registered [ 0.827473] acpiphp: Slot [9] registered [ 0.828966] acpiphp: Slot [10] registered [ 0.830421] acpiphp: Slot [11] registered [ 0.832128] acpiphp: Slot [12] registered [ 0.833061] acpiphp: Slot [13] registered [ 0.834055] acpiphp: Slot [14] registered [ 0.835545] acpiphp: Slot [15] registered [ 0.837278] acpiphp: Slot [16] registered [ 0.839032] acpiphp: Slot [17] registered [ 0.840347] acpiphp: Slot [18] registered [ 0.842121] acpiphp: Slot [19] registered [ 0.843806] acpiphp: Slot [20] registered [ 0.845443] acpiphp: Slot [21] registered [ 0.847199] acpiphp: Slot [22] registered [ 0.849050] acpiphp: Slot [23] registered [ 0.850701] acpiphp: Slot [24] registered [ 0.852408] acpiphp: Slot [25] registered [ 0.854155] acpiphp: Slot [26] registered [ 0.855678] acpiphp: Slot [27] registered [ 0.856950] acpiphp: Slot [28] registered [ 0.858686] acpiphp: Slot [29] registered [ 0.860473] acpiphp: Slot [30] registered [ 0.862299] acpiphp: Slot [31] registered [ 0.864017] PCI host bridge to bus 0000:00 [ 0.865318] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.867881] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.870407] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.873408] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.876276] pci_bus 0000:00: root bus resource [mem 0x140000000-0x1bfffffff window] [ 0.879394] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.881491] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.882477] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.883643] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.891304] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.894535] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.897034] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.899976] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.903039] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.906262] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.907098] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.909252] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.912310] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.914444] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.925414] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.927325] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.928671] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.941621] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.952900] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.969486] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.974961] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.979925] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.984461] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 1.009190] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 1.017043] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 1.022999] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 1.026379] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 1.039040] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 1.048642] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 1.052803] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 1.059224] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 1.074048] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 1.085444] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 1.089552] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 1.094571] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 1.109066] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 1.116738] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 1.120761] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 1.124787] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.138207] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 1.145512] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11) [ 1.148046] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11) [ 1.150707] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11) [ 1.152485] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11) [ 1.154049] ACPI: PCI Interrupt Link [LNKS] (IRQs *9) [ 1.156364] vgaarb: loaded [ 1.157513] SCSI subsystem initialized [ 1.158824] ACPI: bus type USB registered [ 1.160175] usbcore: registered new interface driver usbfs [ 1.161966] usbcore: registered new interface driver hub [ 1.163810] usbcore: registered new device driver usb [ 1.165846] PCI: Using ACPI for IRQ routing [ 1.166690] PCI: pci_cache_line_size set to 64 bytes [ 1.166943] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] [ 1.166947] e820: reserve RAM buffer [mem 0xbffce000-0xbfffffff] [ 1.166949] e820: reserve RAM buffer [mem 0x13ee00000-0x13fffffff] [ 1.167258] NetLabel: Initializing [ 1.167960] NetLabel: domain hash size = 128 [ 1.169075] NetLabel: protocols = UNLABELED CIPSOv4 [ 1.170216] NetLabel: unlabeled traffic allowed by default [ 1.172063] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.173346] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.178234] amd_nb: Cannot enumerate AMD northbridges [ 1.179523] Switched to clocksource kvm-clock [ 1.195168] pnp: PnP ACPI init [ 1.196083] ACPI: bus type PNP registered [ 1.197143] pnp 00:00: Plug and Play ACPI device, IDs PNP0303 (active) [ 1.197196] pnp 00:01: Plug and Play ACPI device, IDs PNP0f13 (active) [ 1.197229] pnp 00:02: [dma 2] [ 1.197254] pnp 00:02: Plug and Play ACPI device, IDs PNP0700 (active) [ 1.197302] pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active) [ 1.197352] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active) [ 1.197433] pnp 00:05: Plug and Play ACPI device, IDs PNP0b00 (active) [ 1.197834] pnp: PnP ACPI: found 6 devices [ 1.199277] ACPI: bus type PNP unregistered [ 1.209307] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.209314] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.209317] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.209319] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.209322] pci_bus 0000:00: resource 8 [mem 0x140000000-0x1bfffffff window] [ 1.209391] NET: Registered protocol family 2 [ 1.210743] TCP established hash table entries: 32768 (order: 6, 262144 bytes) [ 1.212611] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes) [ 1.215234] TCP: Hash tables configured (established 32768 bind 32768) [ 1.216680] TCP: reno registered [ 1.217704] UDP hash table entries: 2048 (order: 5, 196608 bytes) [ 1.219510] UDP-Lite hash table entries: 2048 (order: 5, 196608 bytes) [ 1.220921] NET: Registered protocol family 1 [ 1.222164] RPC: Registered named UNIX socket transport module. [ 1.224087] RPC: Registered udp transport module. [ 1.225545] RPC: Registered tcp transport module. [ 1.226816] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.228173] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.229470] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.230962] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.232263] PCI: CLS 0 bytes, default 64 [ 1.232375] Unpacking initramfs... [ 2.563602] debug: unmapping init [mem 0xffff8800bc2e2000-0xffff8800bffbffff] [ 2.566539] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.568557] software IO TLB [mem 0xb82e2000-0xbc2e2000] (64MB) mapped at [ffff8800b82e2000-ffff8800bc2e1fff] [ 2.571425] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 10737418240 ms ovfl timer [ 2.573795] RAPL PMU: hw unit of domain pp0-core 2^-0 Joules [ 2.574996] RAPL PMU: hw unit of domain package 2^-0 Joules [ 2.576748] RAPL PMU: hw unit of domain dram 2^-0 Joules [ 2.580130] cryptomgr_test (51) used greatest stack depth: 14128 bytes left [ 2.580721] futex hash table entries: 1024 (order: 4, 65536 bytes) [ 2.580766] Initialise system trusted keyring [ 2.611688] HugeTLB registered 1 GB page size, pre-allocated 0 pages [ 2.614092] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 2.619159] zpool: loaded [ 2.620002] zbud: loaded [ 2.621062] VFS: Disk quotas dquot_6.6.0 [ 2.622134] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 2.625029] NFS: Registering the id_resolver key type [ 2.626844] Key type id_resolver registered [ 2.628289] Key type id_legacy registered [ 2.629840] nfs4filelayout_init: NFSv4 File Layout Driver Registering... [ 2.632865] Key type big_key registered [ 2.634010] SELinux: Registering netfilter hooks [ 2.636921] cryptomgr_test (57) used greatest stack depth: 13968 bytes left [ 2.639203] cryptomgr_test (60) used greatest stack depth: 13664 bytes left [ 2.639921] NET: Registered protocol family 38 [ 2.639931] Key type asymmetric registered [ 2.639934] Asymmetric key parser 'x509' registered [ 2.640045] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250) [ 2.640146] io scheduler noop registered [ 2.640150] io scheduler deadline registered (default) [ 2.640221] io scheduler cfq registered [ 2.640225] io scheduler mq-deadline registered [ 2.640229] io scheduler kyber registered [ 2.642642] pci_hotplug: PCI Hot Plug PCI Core version: 0.5 [ 2.642653] pciehp: PCI Express Hot Plug Controller Driver version: 0.4 [ 2.656649] intel_idle: does not run on family 6 model 62 [ 2.656803] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.658912] ACPI: Power Button [PWRF] [ 2.659861] GHES: HEST is not enabled! [ 2.712608] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10 [ 2.766924] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 11 [ 2.893701] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11 [ 2.943914] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 10 [ 3.073338] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.102679] 00:03: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A [ 3.131129] 00:04: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A [ 3.134254] Non-volatile memory driver v1.3 [ 3.135559] Linux agpgart interface v0.103 [ 3.136946] crash memory driver: version 1.1 [ 3.138645] nbd: registered device at major 43 [ 3.142975] virtio-pci 0000:00:05.0: irq 24 for MSI/MSI-X [ 3.143007] virtio-pci 0000:00:05.0: irq 25 for MSI/MSI-X [ 3.143031] virtio-pci 0000:00:05.0: irq 26 for MSI/MSI-X [ 3.143054] virtio-pci 0000:00:05.0: irq 27 for MSI/MSI-X [ 3.143094] virtio-pci 0000:00:05.0: irq 28 for MSI/MSI-X [ 3.150493] virtio_blk virtio1: [vda] 59928 512-byte logical blocks (30.6 MB/29.2 MiB) [ 3.156153] virtio-pci 0000:00:06.0: irq 29 for MSI/MSI-X [ 3.156240] virtio-pci 0000:00:06.0: irq 30 for MSI/MSI-X [ 3.156287] virtio-pci 0000:00:06.0: irq 31 for MSI/MSI-X [ 3.156327] virtio-pci 0000:00:06.0: irq 32 for MSI/MSI-X [ 3.156367] virtio-pci 0000:00:06.0: irq 33 for MSI/MSI-X [ 3.164193] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.169918] virtio-pci 0000:00:07.0: irq 34 for MSI/MSI-X [ 3.169953] virtio-pci 0000:00:07.0: irq 35 for MSI/MSI-X [ 3.169976] virtio-pci 0000:00:07.0: irq 36 for MSI/MSI-X [ 3.170011] virtio-pci 0000:00:07.0: irq 37 for MSI/MSI-X [ 3.170054] virtio-pci 0000:00:07.0: irq 38 for MSI/MSI-X [ 3.174927] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.181378] virtio-pci 0000:00:08.0: irq 39 for MSI/MSI-X [ 3.181411] virtio-pci 0000:00:08.0: irq 40 for MSI/MSI-X [ 3.181439] virtio-pci 0000:00:08.0: irq 41 for MSI/MSI-X [ 3.181461] virtio-pci 0000:00:08.0: irq 42 for MSI/MSI-X [ 3.181484] virtio-pci 0000:00:08.0: irq 43 for MSI/MSI-X [ 3.187395] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.193664] virtio-pci 0000:00:09.0: irq 44 for MSI/MSI-X [ 3.193706] virtio-pci 0000:00:09.0: irq 45 for MSI/MSI-X [ 3.193756] virtio-pci 0000:00:09.0: irq 46 for MSI/MSI-X [ 3.193832] virtio-pci 0000:00:09.0: irq 47 for MSI/MSI-X [ 3.193872] virtio-pci 0000:00:09.0: irq 48 for MSI/MSI-X [ 3.200666] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.207674] virtio-pci 0000:00:0a.0: irq 49 for MSI/MSI-X [ 3.207723] virtio-pci 0000:00:0a.0: irq 50 for MSI/MSI-X [ 3.207766] virtio-pci 0000:00:0a.0: irq 51 for MSI/MSI-X [ 3.207807] virtio-pci 0000:00:0a.0: irq 52 for MSI/MSI-X [ 3.207859] virtio-pci 0000:00:0a.0: irq 53 for MSI/MSI-X [ 3.213398] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.218085] rdac: device handler registered [ 3.219272] hp_sw: device handler registered [ 3.220090] emc: device handler registered [ 3.221050] libphy: Fixed MDIO Bus: probed [ 3.223698] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 3.225869] ehci-pci: EHCI PCI platform driver [ 3.227481] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 3.229958] ohci-pci: OHCI PCI platform driver [ 3.231216] uhci_hcd: USB Universal Host Controller Interface driver [ 3.233352] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.237114] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.238774] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.240751] mousedev: PS/2 mouse device common for all mice [ 3.243157] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.246452] rtc_cmos 00:05: RTC can wake from S4 [ 3.248568] rtc_cmos 00:05: rtc core: registered rtc_cmos as rtc0 [ 3.251142] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.254465] hidraw: raw HID events driver (C) Jiri Kosina [ 3.255904] usbcore: registered new interface driver usbhid [ 3.257136] usbhid: USB HID core driver [ 3.258475] drop_monitor: Initializing network drop monitor service [ 3.260059] Netfilter messages via NETLINK v0.30. [ 3.261026] TCP: cubic registered [ 3.262022] Initializing XFRM netlink socket [ 3.263461] NET: Registered protocol family 10 [ 3.264899] NET: Registered protocol family 17 [ 3.266161] Key type dns_resolver registered [ 3.267543] mce: Using 10 MCE banks [ 3.268536] Loading compiled-in X.509 certificates [ 3.270362] Loaded X.509 cert 'Magrathea: Glacier signing key: e34d0e1b7fcf5b414cce75d36d8482945c781ed6' [ 3.273447] registered taskstats version 1 [ 3.276578] modprobe (71) used greatest stack depth: 13456 bytes left [ 3.280910] Key type trusted registered [ 3.285500] Key type encrypted registered [ 3.286773] IMA: No TPM chip found, activating TPM-bypass! (rc=-19) [ 3.290421] BERT: Boot Error Record Table support is disabled. Enable it by using bert_enable as kernel parameter. [ 3.293449] rtc_cmos 00:05: setting system clock to 2024-05-10 12:06:26 UTC (1715342786) [ 3.295854] debug: unmapping init [mem 0xffffffff81da0000-0xffffffff82018fff] [ 3.298159] Write protecting the kernel read-only data: 12288k [ 3.300240] debug: unmapping init [mem 0xffff8800017fe000-0xffff8800017fffff] [ 3.302235] debug: unmapping init [mem 0xffff880001b9b000-0xffff880001bfffff] [ 3.309754] random: systemd: uninitialized urandom read (16 bytes read) [ 3.313042] random: systemd: uninitialized urandom read (16 bytes read) [ 3.315171] random: systemd: uninitialized urandom read (16 bytes read) [ 3.319140] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN) [ 3.325578] systemd[1]: Detected virtualization kvm. [ 3.327305] systemd[1]: Detected architecture x86-64. [ 3.329248] systemd[1]: Running in initial RAM disk. [ 3.333365] systemd[1]: No hostname configured. [ 3.334775] systemd[1]: Set hostname to . [ 3.336452] random: systemd: uninitialized urandom read (16 bytes read) [ 3.338439] systemd[1]: Initializing machine ID from random generator. [ 3.379815] dracut-rootfs-g (86) used greatest stack depth: 13264 bytes left [ 3.383147] random: systemd: uninitialized urandom read (16 bytes read) [ 3.385507] random: systemd: uninitialized urandom read (16 bytes read) [ 3.387990] random: systemd: uninitialized urandom read (16 bytes read) [ 3.389975] random: systemd: uninitialized urandom read (16 bytes read) [ 3.393096] random: systemd: uninitialized urandom read (16 bytes read) [ 3.395083] random: systemd: uninitialized urandom read (16 bytes read) [ 3.404800] systemd[1]: Reached target Swap. [ 3.407244] systemd[1]: Reached target Timers. [ 3.409488] systemd[1]: Reached target Local File Systems. [ 3.412410] systemd[1]: Created slice Root Slice. [ 3.414950] systemd[1]: Listening on Journal Socket. [ 3.417683] systemd[1]: Listening on udev Control Socket. [ 3.421759] systemd[1]: Listening on udev Kernel Socket. [ 3.424605] systemd[1]: Reached target Sockets. [ 3.427921] systemd[1]: Created slice System Slice. [ 3.433080] systemd[1]: Starting Setup Virtual Console... [ 3.437588] systemd[1]: Starting Journal Service... [ 3.441922] systemd[1]: Starting dracut cmdline hook... [ 3.447191] systemd[1]: Starting Load Kernel Modules... [ 3.452933] systemd[1]: Starting Create list of required static device nodes for the current kernel... [ 3.458101] systemd[1]: Reached target Slices. [ 3.462109] systemd[1]: Started Setup Virtual Console. [ 3.468164] systemd[1]: Started Load Kernel Modules. [ 3.471338] systemd[1]: Started Create list of required static device nodes for the current kernel. [ 3.476044] systemd[1]: Started Journal Service. [ 3.488502] random: fast init done [ 3.580613] tsc: Refined TSC clocksource calibration: 2399.977 MHz [ 3.954792] libata version 3.00 loaded. [ 3.959430] ata_piix 0000:00:01.1: version 2.13 [ 3.977134] scsi host0: ata_piix [ 3.985036] scsi host1: ata_piix [ 3.986875] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 3.989116] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 4.042161] ip (317) used greatest stack depth: 12592 bytes left [ 4.083132] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input2 [ 4.095888] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready [ 4.098193] ip (344) used greatest stack depth: 12464 bytes left [ 4.224718] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [ 7.159560] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ 7.564152] systemd-journald[102]: Received SIGTERM from PID 1 (systemd). [ 7.811118] SELinux: Disabled at runtime. [ 7.812699] SELinux: Unregistering netfilter hooks [ 7.874116] ip_tables: (C) 2000-2006 Netfilter Core Team [ 7.876907] systemd[1]: Inserted module 'ip_tables' [ 8.413008] systemd-journald[565]: Received request to flush runtime journal from PID 1 [ 8.589068] input: PC Speaker as /devices/platform/pcspkr/input/input3 [ 8.613324] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 8.668868] cryptd: max_cpu_qlen set to 1000 [ 8.695356] AVX version of gcm_enc/dec engaged. [ 8.697544] AES CTR mode by8 optimization enabled [ 8.708319] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 8.725724] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ 8.733958] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni) [ 8.734016] alg: No test for __generic-gcm-aes-aesni (__driver-generic-gcm-aes-aesni) [ 8.853117] EDAC MC: Ver: 3.0.0 [ 8.860089] EDAC sbridge: Seeking for: PCI ID 8086:0ea0 [ 8.860100] EDAC sbridge: Ver: 1.1.2 [ 10.333705] mount.nfs (770) used greatest stack depth: 10704 bytes left [ 22.290087] libcfs: loading out-of-tree module taints kernel. [ 22.292680] libcfs: module verification failed: signature and/or required key missing - tainting kernel [ 22.319316] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 26.974448] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 27.186810] alg: No test for adler32 (adler32-zlib) [ 27.937588] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 28.062706] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 28.220755] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 28.222079] LNet: Accept secure, port 988 [ 29.764674] Key type lgssc registered [ 30.057311] Lustre: Echo OBD driver; http://www.lustre.org/ [ 30.479896] icp: module license 'CDDL' taints kernel. [ 30.481065] Disabling lock debugging due to kernel taint [ 32.995190] ZFS: Loaded module v0.8.6-1, ZFS pool version 5000, ZFS filesystem version 5 [ 35.048396] vdc: vdc1 vdc9 [ 37.517487] vde: vde1 vde9 [ 39.936180] vdf: vdf1 vdf9 [ 43.490210] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 45.087365] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 46.204501] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 46.274401] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 46.310956] Lustre: lustre-MDT0000: new disk, initializing [ 46.377796] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 46.392515] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 46.398084] random: crng init done [ 46.445239] mount.lustre (6891) used greatest stack depth: 10240 bytes left [ 47.177335] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 50.359476] Lustre: lustre-OST0000: new disk, initializing [ 50.360845] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 50.362322] Lustre: Skipped 1 previous similar message [ 50.375960] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 51.617952] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 53.306355] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 53.309996] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 53.334295] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 54.705846] Lustre: lustre-OST0001: new disk, initializing [ 54.707310] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 54.722610] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 56.050277] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 60.737698] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 60.740264] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 60.771495] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 60.841364] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 64.322524] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 69.942862] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing check_logdir /tmp/testlogs/ [ 70.735584] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing yml_node [ 71.675464] Lustre: DEBUG MARKER: Client: 2.15.63.2 [ 72.285640] Lustre: DEBUG MARKER: MDS: 2.15.63.2 [ 72.867722] Lustre: DEBUG MARKER: OSS: 2.15.63.2 [ 73.231693] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri May 10 08:07:35 EDT 2024 [ 76.682969] Lustre: DEBUG MARKER: excepting tests: 55 63 67 32newtarball 110 [ 77.013877] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 90.752897] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 90.756886] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 90.758733] Lustre: Skipped 1 previous similar message [ 93.824725] Lustre: server umount lustre-MDT0000 complete [ 94.841192] LustreError: 6963:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1715342878 with bad export cookie 8409075976544279476 [ 94.842427] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 94.847972] LustreError: 6963:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 100.850616] Lustre: server umount lustre-OST0000 complete [ 107.905867] Lustre: server umount lustre-OST0001 complete [ 109.288673] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 110.721969] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 112.661120] vdc: vdc1 vdc9 [ 114.861366] vde: vde1 vde9 [ 117.224628] vdf: vdf1 vdf9 [ 119.512343] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 119.532998] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 119.565781] Lustre: lustre-MDT0000: new disk, initializing [ 119.629697] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 119.646678] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 119.693010] mount.lustre (18135) used greatest stack depth: 9632 bytes left [ 120.409913] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 122.665794] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 124.115226] Lustre: lustre-OST0000: new disk, initializing [ 124.116876] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 124.118277] Lustre: Skipped 1 previous similar message [ 124.137236] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 125.329913] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 125.454314] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 125.458864] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 125.486042] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 127.560889] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 128.313751] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 128.357613] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 130.433281] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 130.439497] Lustre: Skipped 1 previous similar message [ 130.442165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 130.889184] Lustre: server umount lustre-OST0000 complete [ 132.000204] Lustre: server umount lustre-MDT0000 complete [ 133.951022] Lustre: DEBUG MARKER: == conf-sanity test 33a: Mount ost with a large index number ========================================================== 08:08:36 (1715342916) [ 139.347726] Lustre: Setting parameter test-33a-MDT0000.mdt.identity_upcall in log test-33a-MDT0000 [ 139.375413] Lustre: ctl-test-33a-MDT0000: No data found on store. Initialize space. [ 139.414067] Lustre: test-33a-MDT0000: new disk, initializing [ 139.493333] Lustre: test-33a-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 139.510636] Lustre: ctl-test-33a-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 140.249463] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 144.646102] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 145.214131] Lustre: cli-test-33a-OST1fff-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1fff:ost] [ 145.238057] Lustre: test-33a-OST1fff-osc-MDT0000: update sequence from 0x11fff0000 to 0x240000400 [ 150.193297] Lustre: test-33a-OST1fff-osc-MDT0000: Connection to test-33a-OST1fff (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 150.196997] Lustre: test-33a-OST1fff: Not available for connect from 0@lo (stopping) [ 152.320224] Lustre: server umount test-33a-OST1fff complete [ 153.425447] Lustre: server umount test-33a-MDT0000 complete [ 157.723135] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 158.140803] Key type lgssc unregistered [ 158.199832] LNet: 31087:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 159.202788] LNet: Removed LNI 192.168.203.126@tcp [ 167.822345] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 167.918565] alg: No test for adler32 (adler32-zlib) [ 168.669601] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 168.725197] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 168.767868] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 168.769424] LNet: Accept secure, port 988 [ 170.292666] Key type lgssc registered [ 170.488317] Lustre: Echo OBD driver; http://www.lustre.org/ [ 171.694044] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 172.871622] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 173.609469] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 174.657116] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 175.906838] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 177.103856] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 178.120842] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 187.905243] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 187.911130] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 192.251186] Lustre: server umount lustre-OST0000 complete [ 193.348037] Lustre: server umount lustre-MDT0000 complete [ 196.099831] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 196.526168] Key type lgssc unregistered [ 196.582794] LNet: 3723:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 197.584852] LNet: Removed LNI 192.168.203.126@tcp [ 204.749434] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 204.879868] alg: No test for adler32 (adler32-zlib) [ 205.631612] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 205.687009] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 205.736111] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 205.738788] LNet: Accept secure, port 988 [ 207.265662] Key type lgssc registered [ 207.455064] Lustre: Echo OBD driver; http://www.lustre.org/ [ 212.826564] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 213.977267] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 214.000365] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 214.035360] Lustre: lustre-MDT0000: new disk, initializing [ 214.104006] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 214.122953] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 214.870340] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 218.011157] Lustre: lustre-OST07c6: new disk, initializing [ 218.013091] Lustre: srv-lustre-OST07c6: No data found on store. Initialize space. [ 218.015716] Lustre: Skipped 1 previous similar message [ 218.038679] Lustre: lustre-OST07c6: Imperative Recovery not enabled, recovery window 60-180 [ 218.069911] mount.lustre (10012) used greatest stack depth: 9552 bytes left [ 219.268555] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 219.543961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:7c6:ost [ 219.547274] Lustre: cli-lustre-OST07c6-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:7c6:ost] [ 219.573417] Lustre: lustre-OST07c6-osc-MDT0000: update sequence from 0x107c60000 to 0x240000400 [ 289.633321] Lustre: lustre-OST07c6-osc-MDT0000: Connection to lustre-OST07c6 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 289.638663] LustreError: 11-0: lustre-OST07c6-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 289.638923] Lustre: lustre-OST07c6: Not available for connect from 0@lo (stopping) [ 293.455637] Lustre: server umount lustre-OST07c6 complete [ 294.548139] Lustre: server umount lustre-MDT0000 complete [ 297.390151] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 297.824640] Key type lgssc unregistered [ 297.892925] LNet: 20093:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 298.894835] LNet: Removed LNI 192.168.203.126@tcp [ 305.276947] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 305.386746] alg: No test for adler32 (adler32-zlib) [ 306.138701] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 306.197822] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 306.242626] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 306.244014] LNet: Accept secure, port 988 [ 307.773654] Key type lgssc registered [ 307.959215] Lustre: Echo OBD driver; http://www.lustre.org/ [ 309.195019] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 310.404344] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 311.150263] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 312.224504] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 313.519505] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 314.691654] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 315.688936] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 320.516136] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:35 to 0x240000400:65) [ 325.521142] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 325.521178] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 325.527237] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 330.529055] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 330.879011] Lustre: server umount lustre-OST0000 complete [ 332.356283] Lustre: server umount lustre-MDT0000 complete [ 335.669542] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 336.114837] Key type lgssc unregistered [ 336.199216] LNet: 24473:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 337.204026] LNet: Removed LNI 192.168.203.126@tcp [ 345.443021] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 345.593122] alg: No test for adler32 (adler32-zlib) [ 346.343577] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 346.400900] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 346.452877] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 346.454419] LNet: Accept secure, port 988 [ 347.984718] Key type lgssc registered [ 348.190103] Lustre: Echo OBD driver; http://www.lustre.org/ [ 349.488818] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 350.690441] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 351.520026] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 352.734229] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 354.399749] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 356.012971] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 357.493827] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 360.390853] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:67 to 0x240000400:97) [ 365.393053] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 365.397453] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 368.634066] Lustre: lustre-MDT0000: Not available for connect from 192.168.203.26@tcp (stopping) [ 370.401018] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 371.459191] Lustre: server umount lustre-MDT0000 complete [ 382.111963] LustreError: 26273:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 192.168.203.26@tcp arrived at 1715343165 with bad export cookie 10676971332225168670 [ 382.117078] LustreError: 26273:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 382.708336] LustreError: 26273:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1715343165 with bad export cookie 10676971332225168341 [ 382.709896] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 382.717160] LustreError: 26273:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 384.756494] Lustre: server umount lustre-OST0000 complete [ 388.596903] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 389.153538] Key type lgssc unregistered [ 389.218937] LNet: 29282:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 390.223849] LNet: Removed LNI 192.168.203.126@tcp [ 397.974707] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 398.126409] alg: No test for adler32 (adler32-zlib) [ 398.877605] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 398.933632] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 398.978108] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 398.979697] LNet: Accept secure, port 988 [ 400.503715] Key type lgssc registered [ 400.694031] Lustre: Echo OBD driver; http://www.lustre.org/ [ 402.202356] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 403.388365] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 404.273727] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 405.420993] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 406.774938] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 408.016763] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 409.033145] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 413.770639] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 418.769186] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 418.772950] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 420.148647] Lustre: lustre-OST0000: Not available for connect from 192.168.203.26@tcp (stopping) [ 422.926551] Lustre: server umount lustre-OST0000 complete [ 440.850318] Lustre: server umount lustre-MDT0000 complete [ 443.664128] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 444.098458] Key type lgssc unregistered [ 444.165895] LNet: 1769:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 445.168805] LNet: Removed LNI 192.168.203.126@tcp [ 452.594719] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 452.753666] alg: No test for adler32 (adler32-zlib) [ 453.504648] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 453.570779] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 453.624279] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 453.625818] LNet: Accept secure, port 988 [ 455.153693] Key type lgssc registered [ 455.344712] Lustre: Echo OBD driver; http://www.lustre.org/ [ 456.648230] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 457.860399] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 458.611472] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 459.668924] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 461.176069] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 462.728289] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 463.957588] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 468.168210] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:131 to 0x240000400:161) [ 471.706573] Lustre: DEBUG MARKER: Set up a fake failnode for the MDS [ 472.579218] Lustre: DEBUG MARKER: Wait for RECONNECT_INTERVAL seconds (10s) [ 479.784824] LNetError: 120-3: Refusing connection from 192.168.203.126 for 127.0.0.2@tcp: No matching NI [ 479.787178] LNetError: 2890:0:(socklnd_cb.c:1795:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 479.789944] LNetError: 11b-b: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 483.015965] Lustre: DEBUG MARKER: conf-sanity.sh test_35a 2024-05-10 8h14m25s [ 483.360873] Lustre: DEBUG MARKER: Stopping the MDT: lustre-MDT0000 [ 485.127316] Lustre: lustre-MDT0000: Not available for connect from 192.168.203.26@tcp (stopping) [ 486.027900] Lustre: server umount lustre-MDT0000 complete [ 487.177851] Lustre: DEBUG MARKER: Restarting the MDT: lustre-MDT0000 [ 488.904954] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 489.054505] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 489.069342] LNetError: 120-3: Refusing connection from 192.168.203.126 for 127.0.0.2@tcp: No matching NI [ 489.073420] LNetError: 2893:0:(socklnd_cb.c:1795:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 489.077833] LNetError: 11b-b: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 489.092753] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 489.126672] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:131 to 0x240000400:193) [ 489.896883] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 490.739218] Lustre: DEBUG MARKER: Wait for df (30366) ... [ 491.122369] Lustre: DEBUG MARKER: done [ 494.065333] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 494.068515] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 494.072888] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 498.016823] Lustre: server umount lustre-OST0000 complete [ 499.378367] Lustre: server umount lustre-MDT0000 complete [ 502.228990] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 502.686217] Key type lgssc unregistered [ 502.746860] LNet: 8549:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 503.750790] LNet: Removed LNI 192.168.203.126@tcp [ 509.973441] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 513.993643] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 514.148328] alg: No test for adler32 (adler32-zlib) [ 514.899670] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 514.964024] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 515.015223] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 515.016721] LNet: Accept secure, port 988 [ 516.544658] Key type lgssc registered [ 516.734158] Lustre: Echo OBD driver; http://www.lustre.org/ [ 518.603844] vdc: vdc1 vdc9 [ 520.815997] vde: vde1 vde9 [ 523.224895] vdf: vdf1 vdf9 [ 525.380255] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 526.544790] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 526.564056] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 526.602132] Lustre: lustre-MDT0000: new disk, initializing [ 526.667743] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 526.683355] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 527.490952] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 529.764320] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 531.300599] Lustre: lustre-OST0000: new disk, initializing [ 531.302860] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 531.305252] Lustre: Skipped 1 previous similar message [ 531.328219] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 532.536291] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 534.776273] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 534.983695] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 534.986332] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 535.011022] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 535.528138] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 535.578976] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 539.969226] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 539.972979] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 542.105164] Lustre: server umount lustre-OST0000 complete [ 543.213109] Lustre: server umount lustre-MDT0000 complete [ 546.748758] Lustre: DEBUG MARKER: == conf-sanity test 35b: Continue reconnection retries, if the active server is busy ========================================================== 08:15:28 (1715343328) [ 548.147729] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 548.907017] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 549.936205] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 551.258229] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 552.639598] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 553.686302] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 561.773703] Lustre: DEBUG MARKER: conf-sanity.sh test_35b 2024-05-10 8h15m43s [ 562.139262] Lustre: DEBUG MARKER: Set up a fake failnode for the MDS [ 563.759007] Lustre: DEBUG MARKER: Injecting EBUSY on MDS [ 564.333570] Lustre: DEBUG MARKER: Creating a test file and stat it [ 564.344211] LustreError: 18645:0:(mdt_handler.c:2320:mdt_getattr_name_lock()) cfs_fail_timeout id 136 sleeping for 40000ms [ 565.618190] LNetError: 120-3: Refusing connection from 192.168.203.126 for 127.0.0.2@tcp: No matching NI [ 565.620875] LNetError: 10630:0:(socklnd_cb.c:1795:ksocknal_recv_hello()) Error -104 reading HELLO from 127.0.0.2 [ 565.625037] LNetError: 11b-b: Connection to 127.0.0.2@tcp at host 127.0.0.2:988 was reset: is it running a compatible version of Lustre and is 127.0.0.2@tcp one of its NIDs? [ 584.353410] Lustre: lustre-MDT0000: Client 332d29bb-fc54-4209-8c87-b2ec1ac21e7d (at 192.168.203.26@tcp) reconnecting [ 604.346626] LustreError: 18645:0:(mdt_handler.c:2320:mdt_getattr_name_lock()) cfs_fail_timeout id 136 awake [ 604.717369] Lustre: DEBUG MARKER: Stop injecting EBUSY on MDS [ 605.266136] Lustre: DEBUG MARKER: done [ 608.337116] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 608.337157] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 608.337431] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 612.278844] Lustre: server umount lustre-OST0000 complete [ 613.387294] Lustre: server umount lustre-MDT0000 complete [ 616.209763] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 616.722038] Key type lgssc unregistered [ 616.782854] LNet: 22472:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 617.785934] LNet: Removed LNI 192.168.203.126@tcp [ 624.661699] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 629.313940] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 629.447701] alg: No test for adler32 (adler32-zlib) [ 630.198651] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 630.250629] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 630.290559] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 630.292061] LNet: Accept secure, port 988 [ 631.814797] Key type lgssc registered [ 632.126938] Lustre: Echo OBD driver; http://www.lustre.org/ [ 634.177401] vdc: vdc1 vdc9 [ 636.452816] vde: vde1 vde9 [ 638.819801] vdf: vdf1 vdf9 [ 641.302918] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 642.459541] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 642.482561] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 642.520659] Lustre: lustre-MDT0000: new disk, initializing [ 642.580172] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 642.592688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 643.646686] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 646.441800] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 648.199593] Lustre: lustre-OST0000: new disk, initializing [ 648.201252] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 648.202760] Lustre: Skipped 1 previous similar message [ 648.223394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 649.082309] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 649.084908] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 649.106407] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 649.512934] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 651.866059] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 652.691033] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 652.738518] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 654.065768] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 654.075363] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 655.266703] Lustre: server umount lustre-OST0000 complete [ 656.407057] Lustre: server umount lustre-MDT0000 complete [ 659.880725] Lustre: DEBUG MARKER: == conf-sanity test 36: df report consistency on OSTs with different block size ========================================================== 08:17:22 (1715343442) [ 660.280416] Lustre: DEBUG MARKER: SKIP: conf-sanity test_36 remote OST [ 662.305224] Lustre: DEBUG MARKER: == conf-sanity test 37: verify set tunables works for symlink device ========================================================== 08:17:24 (1715343444) [ 662.657539] Lustre: DEBUG MARKER: SKIP: conf-sanity test_37 ldiskfs only test [ 664.901505] Lustre: DEBUG MARKER: == conf-sanity test 38: MDS recreates missing lov_objid file from OST data ========================================================== 08:17:26 (1715343446) [ 666.364283] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 667.179245] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 668.233317] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 669.619573] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 670.803488] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 671.827273] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 679.535604] Lustre: DEBUG MARKER: copying 10 files to /mnt/lustre/d38.conf-sanity [ 681.617556] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 681.626838] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 682.842466] Lustre: server umount lustre-MDT0000 complete [ 683.923890] Lustre: DEBUG MARKER: delete lov_objid file on MDS [ 689.686808] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 689.826614] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 689.864957] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000bd0 [ 690.759173] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 691.992421] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 699.449008] Lustre: server umount lustre-MDT0000 complete [ 704.604464] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 704.734653] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 704.761184] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000bd0:3 to 0x240000bd0:33) [ 705.577634] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 707.035389] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 714.848720] Lustre: server umount lustre-MDT0000 complete [ 719.012129] Lustre: DEBUG MARKER: files compared the same [ 719.517589] LustreError: 1420:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1715343502 with bad export cookie 8946412831341925300 [ 719.519121] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 719.524882] LustreError: 1420:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 719.583062] Lustre: server umount lustre-OST0000 complete [ 723.439623] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 724.059150] Key type lgssc unregistered [ 724.148259] LNet: 10913:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 725.154015] LNet: Removed LNI 192.168.203.126@tcp [ 734.451725] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 734.550994] alg: No test for adler32 (adler32-zlib) [ 735.301588] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 735.373739] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 735.449217] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 735.453829] LNet: Accept secure, port 988 [ 737.001624] Key type lgssc registered [ 737.275563] Lustre: Echo OBD driver; http://www.lustre.org/ [ 738.655505] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 739.852380] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 740.589139] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug +malloc all [ 741.677441] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 743.023151] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 744.216464] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug +malloc all [ 745.441210] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 750.020352] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000bd0:35 to 0x240000bd0:65) [ 755.025205] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 755.025268] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 755.037855] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 759.525487] Lustre: server umount lustre-OST0000 complete [ 760.891802] Lustre: server umount lustre-MDT0000 complete [ 764.004836] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 764.464412] Key type lgssc unregistered [ 764.521776] LNet: 15591:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 765.524110] LNet: Removed LNI 192.168.203.126@tcp [ 774.853908] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 775.036653] alg: No test for adler32 (adler32-zlib) [ 775.787640] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 775.849093] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 775.916103] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 775.919777] LNet: Accept secure, port 988 [ 777.468709] Key type lgssc registered [ 777.795453] Lustre: Echo OBD driver; http://www.lustre.org/ [ 779.418292] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 796.512804] LustreError: 13a-8: Failed to get MGS log lustre-sptlrpc and no local copy. [ 832.517692] LustreError: 17408:0:(mgc_request.c:253:do_config_log_add()) MGC192.168.203.126@tcp: failed processing log, type 4: rc = -110 [ 862.536642] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 864.244069] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 865.714315] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 867.866388] Lustre: *** cfs_fail_loc=706, val=0*** [ 868.005823] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 868.033647] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000bd0:35 to 0x240000bd0:97) [ 868.871010] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 869.983030] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 872.756135] Lustre: server umount lustre-OST0000 complete [ 884.012921] Lustre: server umount lustre-MDT0000 complete [ 885.944820] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 886.442663] Key type lgssc unregistered [ 886.498869] LNet: 20292:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 887.501987] LNet: Removed LNI 192.168.203.126@tcp [ 896.464965] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 896.621214] alg: No test for adler32 (adler32-zlib) [ 897.372655] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 897.426183] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 897.473683] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 897.475182] LNet: Accept secure, port 988 [ 899.002662] Key type lgssc registered [ 899.200845] Lustre: Echo OBD driver; http://www.lustre.org/ [ 900.656967] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 902.819016] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 904.925985] Lustre: 22921:0:(mgc_request_server.c:553:mgc_llog_local_copy()) MGC192.168.203.126@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 904.931308] Lustre: 22921:0:(mgc_request_server.c:597:mgc_process_server_cfg_log()) MGC192.168.203.126@tcp: local log lustre-sptlrpc are not valid and/or remote logs are not accessbile rc = -2 [ 904.988722] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 905.009980] mount.lustre (22921) used greatest stack depth: 9520 bytes left [ 906.673068] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 908.200482] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 908.203936] Lustre: MGS: Client e99ca5e3-eb59-491e-a821-04552b23f456 (at 0@lo) reconnecting [ 908.330668] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 908.368014] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000bd0:35 to 0x240000bd0:129) [ 909.330503] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 918.321478] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 918.325955] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 918.334202] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 920.337260] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 923.044710] Lustre: server umount lustre-OST0000 complete [ 924.411417] Lustre: server umount lustre-MDT0000 complete [ 925.730845] Lustre: server umount MGS complete [ 930.380426] Lustre: DEBUG MARKER: == conf-sanity test 41b: mount mds with --nosvc and --nomgs on first mount ========================================================== 08:21:52 (1715343712) [ 937.913547] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 939.801871] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 942.724237] vdc: vdc1 vdc9 [ 946.041270] vde: vde1 vde9 [ 949.485966] vdf: vdf1 vdf9 [ 953.323371] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 956.905881] Lustre: 31794:0:(mgc_request_server.c:553:mgc_llog_local_copy()) MGC192.168.203.126@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 956.909498] Lustre: 31794:0:(mgc_request_server.c:597:mgc_process_server_cfg_log()) MGC192.168.203.126@tcp: local log lustre-sptlrpc are not valid and/or remote logs are not accessbile rc = -2 [ 956.921651] Lustre: lustre-OST0000: new disk, initializing [ 956.923343] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 956.946576] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 958.685510] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 961.567247] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 962.819907] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 962.827625] Lustre: MGS: Client c79396ff-72e0-47ea-a2d6-4a15ea7c85d8 (at 0@lo) reconnecting [ 962.922635] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 962.930425] LustreError: 30949:0:(mgs_handler.c:264:mgs_revoke_lock()) MGS: can't take cfg lock for 0x65727473756c/0x0 : rc = -11 [ 962.956479] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 962.997199] Lustre: lustre-MDT0000: new disk, initializing [ 963.077272] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 963.094389] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 963.147983] Lustre: srv-lustre-OST0000: Waiting to contact MDT0000 to allocate super-sequence: rc = -115 [ 964.082814] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 964.151182] Lustre: 1064:0:(client.c:1513:after_reply()) @@@ resending request on EINPROGRESS req@ffff880090c3f480 x1798668214608576/t0(0) o700->lustre-OST0000-osc-MDT0000@0@lo:31/4 lens 264/248 e 0 to 0 dl 1715343763 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'' uid:0 gid:0 [ 969.177729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 969.183811] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 969.211647] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 982.513820] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 982.521574] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 987.521343] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 988.172621] Lustre: server umount lustre-OST0000 complete [ 995.688922] Lustre: server umount lustre-MDT0000 complete [ 996.843121] Lustre: server umount MGS complete [ 1000.915233] Lustre: DEBUG MARKER: == conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one ========================================================== 08:23:02 (1715343782) [ 1005.979814] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1006.618912] Key type lgssc unregistered [ 1006.697154] LNet: 4323:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1007.702160] LNet: Removed LNI 192.168.203.126@tcp [ 1013.410246] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1013.606088] alg: No test for adler32 (adler32-zlib) [ 1014.356593] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1014.425483] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1014.491401] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1014.494325] LNet: Accept secure, port 988 [ 1016.036704] Key type lgssc registered [ 1016.387168] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1018.396763] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1019.734004] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1020.725889] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1021.898412] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1023.544304] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1025.395216] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1027.014618] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1029.539692] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:3 to 0x240000400:33) [ 1039.553726] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1039.553804] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1039.567460] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1041.551104] Lustre: server umount lustre-OST0000 complete [ 1043.181701] Lustre: server umount lustre-MDT0000 complete [ 1046.920569] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1047.693654] Key type lgssc unregistered [ 1047.781085] LNet: 8291:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1048.784982] LNet: Removed LNI 192.168.203.126@tcp [ 1054.472099] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1054.621374] alg: No test for adler32 (adler32-zlib) [ 1055.372639] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1055.426639] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1055.471979] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1055.473442] LNet: Accept secure, port 988 [ 1056.997638] Key type lgssc registered [ 1057.238499] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1059.428505] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1059.428750] LustreError: 9504:0:(tgt_mount.c:2178:server_fill_super()) cfs_race id 716 sleeping [ 1059.436698] Lustre: Skipped 1 previous similar message [ 1059.438914] LustreError: 9503:0:(tgt_mount.c:2178:server_fill_super()) cfs_fail_race id 716 waking [ 1059.443101] LustreError: 9504:0:(tgt_mount.c:2178:server_fill_super()) cfs_fail_race id 716 awake: rc=4986 [ 1059.448107] LustreError: 9504:0:(tgt_mount.c:2190:server_fill_super()) Unable to start osd on lustre-mdt1/mdt1: -114 [ 1059.454043] LustreError: 9504:0:(super25.c:189:lustre_fill_super()) llite: Unable to mount : rc = -114 [ 1060.737708] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1061.864758] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing lsmod [ 1063.211225] LustreError: 10297:0:(tgt_mount.c:2178:server_fill_super()) cfs_race id 716 sleeping [ 1063.211869] LustreError: 10298:0:(tgt_mount.c:2178:server_fill_super()) cfs_fail_race id 716 waking [ 1063.222649] LustreError: 10297:0:(tgt_mount.c:2178:server_fill_super()) cfs_fail_race id 716 awake: rc=5000 [ 1063.228548] LustreError: 10297:0:(tgt_mount.c:2190:server_fill_super()) Unable to start osd on lustre-ost1/ost1: -114 [ 1063.236277] LustreError: 10297:0:(super25.c:189:lustre_fill_super()) llite: Unable to mount : rc = -114 [ 1063.350448] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1064.466457] Lustre: server umount lustre-MDT0000 complete [ 1065.873115] LustreError: 9526:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1715343849 with bad export cookie 2474249674622096382 [ 1065.877330] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1075.955993] Lustre: server umount lustre-OST0000 complete [ 1078.513929] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1079.670114] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1081.249712] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1083.245401] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1085.036695] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1086.580680] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1088.240713] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:3 to 0x240000400:65) [ 1096.759239] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 1096.764108] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1096.771809] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1098.257436] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1099.863867] Lustre: server umount lustre-OST0000 complete [ 1101.355430] Lustre: server umount lustre-MDT0000 complete [ 1104.844766] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1105.575703] Key type lgssc unregistered [ 1105.661240] LNet: 14329:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1106.665895] LNet: Removed LNI 192.168.203.126@tcp [ 1116.767321] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1116.941211] alg: No test for adler32 (adler32-zlib) [ 1117.698647] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1117.763082] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1117.833583] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1117.837460] LNet: Accept secure, port 988 [ 1119.388820] Key type lgssc registered [ 1119.722681] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1121.689184] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1122.897580] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1123.818989] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1125.095000] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1126.781043] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1128.200947] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1129.657458] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1133.774600] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:67 to 0x240000400:97) [ 1137.432892] Lustre: Setting parameter lustre-client.llite.some_wrong_param in log lustre-client [ 1137.997477] Lustre: Modifying parameter lustre-client.llite.some_wrong_param in log lustre-client [ 1139.823113] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 1139.825044] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1139.828204] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1143.794675] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1144.947634] Lustre: server umount lustre-OST0000 complete [ 1146.569907] Lustre: server umount lustre-MDT0000 complete [ 1150.232637] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1151.000203] Key type lgssc unregistered [ 1151.087135] LNet: 19168:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1152.092052] LNet: Removed LNI 192.168.203.126@tcp [ 1157.829783] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1158.013764] alg: No test for adler32 (adler32-zlib) [ 1158.765590] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1158.827821] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1158.881051] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1158.884845] LNet: Accept secure, port 988 [ 1160.434773] Key type lgssc registered [ 1160.783946] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1162.484682] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1163.791514] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1164.776868] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1166.342927] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1168.320542] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1170.026336] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1171.609665] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1173.313700] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 1181.806874] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 1181.811599] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1181.821257] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1183.329260] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1186.270860] Lustre: server umount lustre-OST0000 complete [ 1187.741450] Lustre: server umount lustre-MDT0000 complete [ 1191.099508] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1191.901982] Key type lgssc unregistered [ 1191.999086] LNet: 23255:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1193.003010] LNet: Removed LNI 192.168.203.126@tcp [ 1203.245975] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1203.455703] alg: No test for adler32 (adler32-zlib) [ 1204.206626] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1204.270761] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1204.317873] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1204.320273] LNet: Accept secure, port 988 [ 1205.845809] Key type lgssc registered [ 1206.189651] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1208.100502] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1209.442312] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1210.472373] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1211.975002] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1213.934479] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1215.760907] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1217.356876] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1218.925550] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:131 to 0x240000400:161) [ 1225.810714] Lustre: Setting parameter lustre-MDT0000.mdt.root_squash in log lustre-MDT0000 [ 1227.139319] Lustre: Setting parameter lustre-MDT0000.mdt.nosquash_nids in log lustre-MDT0000 [ 1227.145214] Lustre: Skipped 1 previous similar message [ 1228.668732] Lustre: Modifying parameter lustre-MDT0000.mdt.root_squash in log lustre-MDT0000 [ 1228.672653] Lustre: Skipped 1 previous similar message [ 1233.997920] Lustre: lustre-MDT0000: nosquash_nids is cleared [ 1234.001937] Lustre: lustre-MDT0000: root_squash is set to 500:500 [ 1236.298138] Lustre: Modifying parameter lustre-MDT0000.mdt.nosquash_nids in log lustre-MDT0000 [ 1236.302058] Lustre: Skipped 1 previous similar message [ 1241.638439] Lustre: lustre-MDT0000: nosquash_nids set to 2@gni 0@lo 192.168.203.26@tcp 192.168.0.[2,10]@tcp [ 1243.953337] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1243.960907] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1245.016093] Lustre: server umount lustre-OST0000 complete [ 1246.590263] Lustre: server umount lustre-MDT0000 complete [ 1250.368486] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1251.125806] Key type lgssc unregistered [ 1251.211200] LNet: 29758:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1252.216049] LNet: Removed LNI 192.168.203.126@tcp [ 1265.754143] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1265.947229] alg: No test for adler32 (adler32-zlib) [ 1266.697636] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1266.776428] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1266.844846] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1266.848015] LNet: Accept secure, port 988 [ 1268.392680] Key type lgssc registered [ 1268.728913] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1270.813238] Lustre: MGS: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1273.032698] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1274.462239] Lustre: server umount MGS complete [ 1279.339235] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1279.963583] Key type lgssc unregistered [ 1280.050166] LNet: 2622:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1281.054125] LNet: Removed LNI 192.168.203.126@tcp [ 1291.180969] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1291.354020] alg: No test for adler32 (adler32-zlib) [ 1292.106574] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1292.166967] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1292.222392] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1292.224446] LNet: Accept secure, port 988 [ 1293.767817] Key type lgssc registered [ 1294.093393] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1296.071934] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1297.347547] Lustre: lustre-MDT0000: root_squash is set to 500:500 [ 1297.353009] Lustre: lustre-MDT0000: nosquash_nids set to 2@gni 0@lo 192.168.203.26@tcp 192.168.0.[2,10]@tcp [ 1297.361393] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1298.051274] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1299.182373] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1301.169150] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1302.880765] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1304.401738] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1307.165499] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:167 to 0x240000400:193) [ 1314.565043] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 1314.570107] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1314.577998] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1317.185032] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1319.417190] Lustre: server umount lustre-OST0000 complete [ 1320.877313] Lustre: server umount lustre-MDT0000 complete [ 1324.617137] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1325.358397] Key type lgssc unregistered [ 1325.441028] LNet: 7447:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1326.446084] LNet: Removed LNI 192.168.203.126@tcp [ 1336.184999] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 1341.232974] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1341.383836] alg: No test for adler32 (adler32-zlib) [ 1342.134564] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1342.206944] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1342.274282] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1342.277225] LNet: Accept secure, port 988 [ 1343.819699] Key type lgssc registered [ 1344.125095] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1346.814594] vdc: vdc1 vdc9 [ 1349.305114] vde: vde1 vde9 [ 1352.763637] vdf: vdf1 vdf9 [ 1355.756891] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1356.910347] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1356.934339] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1356.970861] Lustre: lustre-MDT0000: new disk, initializing [ 1357.028407] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1357.042592] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1358.045885] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1360.892091] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1362.744133] Lustre: lustre-OST0000: new disk, initializing [ 1362.746089] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1362.747837] Lustre: Skipped 1 previous similar message [ 1362.765677] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1364.330853] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1364.334835] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1364.361250] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1364.429883] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1367.324336] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1368.449291] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1368.534830] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1369.297625] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1369.305586] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1371.303757] Lustre: server umount lustre-OST0000 complete [ 1372.946307] Lustre: server umount lustre-MDT0000 complete [ 1375.724656] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1376.862391] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1378.430590] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1380.451690] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1382.065297] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1383.605883] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1384.806773] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1385.912777] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1389.215387] Lustre: lustre-OST0001: new disk, initializing [ 1389.218564] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1389.247970] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1391.071090] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1393.245825] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1393.253156] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1393.291316] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1394.239171] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1394.316631] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1395.177470] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b60a3000.ost_server_uuid 50 [ 1395.699097] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8800b60a3000.ost_server_uuid in FULL state after 0 sec [ 1398.225487] LustreError: 11-0: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1398.225759] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1398.226440] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1403.233546] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1403.526919] Lustre: server umount lustre-OST0001 complete [ 1405.473257] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1405.478050] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1405.485540] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1407.044442] Lustre: server umount lustre-OST0000 complete [ 1408.653160] Lustre: server umount lustre-MDT0000 complete [ 1414.511982] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1415.252378] Key type lgssc unregistered [ 1415.338162] LNet: 22920:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1416.342976] LNet: Removed LNI 192.168.203.126@tcp [ 1425.341294] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 1430.438668] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1430.620075] alg: No test for adler32 (adler32-zlib) [ 1431.370634] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1431.428915] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1431.489795] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1431.492970] LNet: Accept secure, port 988 [ 1433.036792] Key type lgssc registered [ 1433.362000] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1436.139264] vdc: vdc1 vdc9 [ 1439.325358] vde: vde1 vde9 [ 1442.793749] vdf: vdf1 vdf9 [ 1445.981815] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1447.172556] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1447.202742] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1447.245242] Lustre: lustre-MDT0000: new disk, initializing [ 1447.301937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1447.315829] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1448.351737] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1451.214018] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1453.375305] Lustre: lustre-OST0000: new disk, initializing [ 1453.378004] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1453.380938] Lustre: Skipped 1 previous similar message [ 1453.410709] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1455.189086] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1455.385823] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1455.392100] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1455.436808] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1458.065076] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1459.255826] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1459.337900] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1460.353789] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1460.361250] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1462.128498] Lustre: server umount lustre-OST0000 complete [ 1463.589237] Lustre: server umount lustre-MDT0000 complete [ 1468.738891] Lustre: DEBUG MARKER: == conf-sanity test 47: server restart does not make client loss lru_resize settings ========================================================== 08:30:50 (1715344250) [ 1472.261747] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 1474.428434] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1477.237196] vdc: vdc1 vdc9 [ 1480.432102] vde: vde1 vde9 [ 1483.884629] vdf: vdf1 vdf9 [ 1487.085737] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1487.109143] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1487.146305] Lustre: lustre-MDT0000: new disk, initializing [ 1487.204422] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1487.217886] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1488.307869] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1490.762349] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1492.130186] Lustre: lustre-OST0000: new disk, initializing [ 1492.132604] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1492.134766] Lustre: Skipped 1 previous similar message [ 1492.154159] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1493.264235] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1495.427169] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1496.287865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1496.290614] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1496.311433] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1502.083198] Lustre: Failing over lustre-OST0000 [ 1502.136818] Lustre: server umount lustre-OST0000 complete [ 1513.632299] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1513.640924] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1513.644947] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 1514.752469] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1515.700605] Lustre: Failing over lustre-MDT0000 [ 1515.793686] Lustre: server umount lustre-MDT0000 complete [ 1516.546144] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 1527.471800] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1527.632076] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_connect to node 0@lo failed: rc = -114 [ 1527.632204] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1527.664823] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1527.704220] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1528.650041] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1530.605149] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects [ 1530.616882] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted. [ 1532.658238] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 192.168.203.126@tcp (at 0@lo) [ 1532.669076] Lustre: lustre-OST0000: Recovery over after 0:17, of 2 clients 2 recovered and 0 were evicted. [ 1532.670832] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:3 to 0x240000400:33) [ 1537.665949] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1537.671354] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1539.360826] Lustre: server umount lustre-OST0000 complete [ 1540.512429] Lustre: server umount lustre-MDT0000 complete [ 1543.227534] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1543.635850] Key type lgssc unregistered [ 1543.694824] LNet: 11042:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1544.696856] LNet: Removed LNI 192.168.203.126@tcp [ 1552.344455] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1552.468319] alg: No test for adler32 (adler32-zlib) [ 1553.219587] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1553.270987] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1553.312658] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1553.314198] LNet: Accept secure, port 988 [ 1554.838671] Key type lgssc registered [ 1555.022466] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1556.277772] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1557.500597] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1558.256742] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1559.294796] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1560.717829] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1561.974271] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1562.935819] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1567.714762] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:3 to 0x240000400:65) [ 1660.919164] Lustre: Failing over lustre-MDT0000 [ 1661.048728] Lustre: server umount lustre-MDT0000 complete [ 1673.491172] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1673.672924] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1673.684071] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1673.718091] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1674.256434] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects [ 1674.295131] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted. [ 1674.321053] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:68 to 0x240000400:97) [ 1674.821338] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1677.807123] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1678.363954] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1678.675307] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 192.168.203.126@tcp (at 0@lo) [ 1679.672685] Lustre: 12178:0:(client.c:2343:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1715344446/real 1715344446] req@ffff88009de11f80 x1798668902763712/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1715344462 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 1684.673623] Lustre: 12176:0:(client.c:2343:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1715344451/real 1715344451] req@ffff88009de12680 x1798668902763840/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1715344467 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 1689.688700] Lustre: 12178:0:(client.c:2343:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1715344456/real 1715344456] req@ffff88009de10700 x1798668902763968/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1715344472 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 1693.697026] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1693.703098] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1696.693046] Lustre: server umount lustre-MDT0000 complete [ 1697.646319] LustreError: 14560:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1715344480 with bad export cookie 16488790614709015075 [ 1697.647977] LustreError: 166-1: MGC192.168.203.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1697.654200] LustreError: 14560:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1703.698316] Lustre: server umount lustre-OST0000 complete [ 1706.412612] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 1708.679392] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1711.564450] vdc: vdc1 vdc9 [ 1714.774124] vde: vde1 vde9 [ 1718.125293] vdf: vdf1 vdf9 [ 1721.546440] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1721.568880] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1721.610278] Lustre: lustre-MDT0000: new disk, initializing [ 1721.683635] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1721.699689] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1722.487996] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1724.647516] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1725.982699] Lustre: lustre-OST0000: new disk, initializing [ 1725.985287] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1725.988039] Lustre: Skipped 1 previous similar message [ 1726.007967] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1727.144775] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1729.661158] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1730.648832] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1730.701051] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1730.703404] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1730.726843] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1731.721136] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1735.697085] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1735.697192] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1735.697498] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1738.352601] Lustre: server umount lustre-OST0000 complete [ 1739.617187] Lustre: server umount lustre-MDT0000 complete [ 1743.821977] Lustre: DEBUG MARKER: == conf-sanity test 49a: check PARAM_SYS_LDLM_TIMEOUT option of mkfs.lustre ========================================================== 08:35:25 (1715344525) [ 1747.497240] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 1748.981907] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1751.254139] vdc: vdc1 vdc9 [ 1753.920054] vde: vde1 vde9 [ 1756.626547] vdf: vdf1 vdf9 [ 1759.589396] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1759.606633] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1759.637984] Lustre: lustre-MDT0000: new disk, initializing [ 1759.694774] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1759.709363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1760.667826] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1763.068658] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1764.698103] Lustre: lustre-OST0000: new disk, initializing [ 1764.700469] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1764.702458] Lustre: Skipped 1 previous similar message [ 1765.485695] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1765.488592] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1765.507196] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1765.922882] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1768.441356] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1770.481409] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1770.486081] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1770.493582] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1771.990623] Lustre: server umount lustre-OST0000 complete [ 1775.218800] Lustre: server umount lustre-MDT0000 complete [ 1779.213952] Lustre: DEBUG MARKER: == conf-sanity test 49b: check PARAM_SYS_LDLM_TIMEOUT option of mkfs.lustre ========================================================== 08:36:01 (1715344561) [ 1782.612346] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 1783.931270] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1786.292044] vdc: vdc1 vdc9 [ 1789.528602] vde: vde1 vde9 [ 1792.962615] vdf: vdf1 vdf9 [ 1796.320354] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1796.342730] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1796.377812] Lustre: lustre-MDT0000: new disk, initializing [ 1796.434643] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1796.436398] Lustre: Skipped 1 previous similar message [ 1796.449088] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1797.542909] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1800.279626] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1804.016987] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1805.563755] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1805.568058] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1805.594822] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1806.547714] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1810.545493] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1810.555172] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1814.404671] Lustre: server umount lustre-OST0000 complete [ 1815.898460] Lustre: server umount lustre-MDT0000 complete [ 1819.080675] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1819.701572] Key type lgssc unregistered [ 1819.787979] LNet: 9998:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1820.792068] LNet: Removed LNI 192.168.203.126@tcp [ 1829.405289] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1829.549476] alg: No test for adler32 (adler32-zlib) [ 1830.300597] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1830.360409] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1830.416953] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1830.419594] LNet: Accept secure, port 988 [ 1831.959651] Key type lgssc registered [ 1832.199422] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1833.543438] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1834.748549] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1835.654235] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1836.938866] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1838.378892] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1839.679158] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1840.704170] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1844.375779] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:3 to 0x240000400:33) [ 1859.393323] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1859.398185] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1859.879190] Lustre: server umount lustre-OST0000 complete [ 1861.252756] Lustre: server umount lustre-MDT0000 complete [ 1864.283206] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 1864.708024] Key type lgssc unregistered [ 1864.774904] LNet: 14721:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1865.777870] LNet: Removed LNI 192.168.203.126@tcp [ 1875.766270] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1875.963360] alg: No test for adler32 (adler32-zlib) [ 1876.713667] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1876.790620] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 1876.842893] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 1876.845113] LNet: Accept secure, port 988 [ 1878.380808] Key type lgssc registered [ 1878.681682] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1880.595636] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1881.942446] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1883.078616] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1884.309656] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1885.827384] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1887.226603] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1888.561337] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1891.822206] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:35 to 0x240000400:65) [ 1896.817256] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1896.821742] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1899.720167] Lustre: lustre-OST0000: Not available for connect from 192.168.203.26@tcp (stopping) [ 1901.825776] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1902.542045] Lustre: server umount lustre-OST0000 complete [ 1903.805701] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state DISCONN osc.lustre-OST0000-osc-ffff8800b6c87000.ost_server_uuid 50 [ 1904.316694] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8800b6c87000.ost_server_uuid in DISCONN state after 0 sec [ 1904.740509] Lustre: DEBUG MARKER: OSCs should all be DISCONN [ 1920.853318] Lustre: server umount lustre-MDT0000 complete [ 1925.885672] Lustre: DEBUG MARKER: == conf-sanity test 50c: lazystatfs one server down ====== 08:38:27 (1715344707) [ 1927.874313] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1928.933218] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1930.448122] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1932.422414] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1934.211324] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1935.761876] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1937.821426] Lustre: lustre-OST0001: new disk, initializing [ 1937.823889] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1937.855589] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1938.851927] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:67 to 0x240000400:97) [ 1939.643822] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1942.510194] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1942.593440] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1942.596243] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1942.623972] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1943.857653] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1943.865298] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1943.874474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1949.461769] Lustre: server umount lustre-OST0000 complete [ 1951.370363] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state DISCONN os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1951.458476] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in DISCONN state after 0 sec [ 1952.578285] LustreError: 137-5: lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1957.585698] LustreError: 137-5: lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1957.593882] LustreError: Skipped 1 previous similar message [ 1962.593409] LustreError: 137-5: lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1962.593709] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1962.594234] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1962.594236] Lustre: Skipped 2 previous similar messages [ 1962.623719] LustreError: Skipped 1 previous similar message [ 1963.477252] Lustre: server umount lustre-OST0001 complete [ 1964.999472] Lustre: server umount lustre-MDT0000 complete [ 1974.273212] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 1976.479266] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 1979.333590] vdc: vdc1 vdc9 [ 1982.581222] vde: vde1 vde9 [ 1986.088202] vdf: vdf1 vdf9 [ 1989.521992] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1989.542404] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1989.580876] Lustre: lustre-MDT0000: new disk, initializing [ 1989.659297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1989.675777] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1990.775711] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1993.652881] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1995.825328] Lustre: lustre-OST0000: new disk, initializing [ 1995.828068] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1995.830741] Lustre: Skipped 1 previous similar message [ 1997.551440] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 1998.752245] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1998.756835] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1998.802427] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1999.939171] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2000.861343] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2000.901677] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2003.729209] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2003.732424] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2007.492675] Lustre: server umount lustre-OST0000 complete [ 2008.920100] Lustre: server umount lustre-MDT0000 complete [ 2012.948192] Lustre: DEBUG MARKER: == conf-sanity test 50d: lazystatfs client/server conn race ========================================================== 08:39:55 (1715344795) [ 2015.136145] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2015.141610] Lustre: Skipped 1 previous similar message [ 2016.268172] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2017.817720] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2021.579053] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2023.147146] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2025.385692] Lustre: lustre-OST0001: new disk, initializing [ 2025.387856] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2027.208811] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2030.127778] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2031.409476] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2031.415556] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2031.496164] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2031.498745] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2031.524694] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2037.106629] Lustre: server umount lustre-OST0000 complete [ 2040.301874] LustreError: 137-5: lustre-OST0000: not available for connect from 192.168.203.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2045.310079] LustreError: 137-5: lustre-OST0000: not available for connect from 192.168.203.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2045.315675] LustreError: Skipped 1 previous similar message [ 2051.505217] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2053.413218] Lustre: server umount lustre-OST0001 complete [ 2054.892489] Lustre: server umount lustre-MDT0000 complete [ 2064.296216] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 2066.518302] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 2069.350313] vdc: vdc1 vdc9 [ 2072.294317] vde: vde1 vde9 [ 2074.529246] vdf: vdf1 vdf9 [ 2076.606472] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 2076.628456] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2076.660269] Lustre: lustre-MDT0000: new disk, initializing [ 2076.721249] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2076.723469] Lustre: Skipped 2 previous similar messages [ 2076.736466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2077.457787] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2079.646915] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2082.236339] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2084.437657] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2084.794014] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2084.796608] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2084.822501] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2085.174818] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2085.216813] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2089.777717] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2089.785394] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2089.788906] Lustre: Skipped 3 previous similar messages [ 2091.749048] Lustre: server umount lustre-OST0000 complete [ 2092.878144] Lustre: server umount lustre-MDT0000 complete [ 2096.936569] Lustre: DEBUG MARKER: == conf-sanity test 50e: normal statfs all servers down == 08:41:19 (1715344879) [ 2100.181835] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 2102.335465] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 2105.280588] vdc: vdc1 vdc9 [ 2108.087764] vde: vde1 vde9 [ 2111.371759] vdf: vdf1 vdf9 [ 2114.618767] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 2114.652047] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2114.653911] Lustre: Skipped 2 previous similar messages [ 2114.693274] Lustre: lustre-MDT0000: new disk, initializing [ 2114.694663] Lustre: Skipped 1 previous similar message [ 2114.809949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2115.862394] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2118.642873] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2121.820692] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2123.296749] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2123.318785] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2124.443575] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2125.189741] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2125.234390] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2128.273666] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2131.906691] Lustre: server umount lustre-OST0000 complete [ 2132.939421] Lustre: server umount lustre-MDT0000 complete [ 2136.268615] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2137.686699] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2140.662264] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2141.660750] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2142.459635] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2145.585040] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 2150.337365] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2152.236598] Lustre: server umount lustre-OST0000 complete [ 2154.056665] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state DISCONN os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2154.137177] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in DISCONN state after 0 sec [ 2177.148878] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2177.150830] Lustre: Skipped 5 previous similar messages [ 2178.864610] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2180.388232] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2180.964063] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 2183.862267] Lustre: server umount lustre-OST0000 complete [ 2185.100538] Lustre: server umount lustre-MDT0000 complete [ 2189.328657] Lustre: DEBUG MARKER: == conf-sanity test 50f: normal statfs one server in down ========================================================== 08:42:51 (1715344971) [ 2191.849753] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2192.955885] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2195.736054] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2197.037499] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2197.799757] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2201.911295] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 2203.317558] Lustre: lustre-OST0001: new disk, initializing [ 2203.319470] Lustre: Skipped 1 previous similar message [ 2203.321726] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2203.324048] Lustre: Skipped 2 previous similar messages [ 2204.557002] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2206.210044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2206.212350] Lustre: Skipped 1 previous similar message [ 2206.213603] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2206.233175] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2206.881796] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2207.646351] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2207.686993] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2211.202121] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2211.208470] Lustre: Skipped 1 previous similar message [ 2211.211191] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2211.214196] Lustre: Skipped 2 previous similar messages [ 2214.201830] Lustre: server umount lustre-OST0001 complete [ 2215.807759] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state DISCONN os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2215.889984] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in DISCONN state after 0 sec [ 2216.005508] LustreError: 137-5: lustre-OST0001: not available for connect from 192.168.203.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2216.012552] LustreError: Skipped 2 previous similar messages [ 2236.045881] LustreError: 137-5: lustre-OST0001: not available for connect from 192.168.203.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2236.049156] LustreError: Skipped 7 previous similar messages [ 2239.520375] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2240.577557] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2242.027532] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 1 sec [ 2244.600265] Lustre: server umount lustre-OST0001 complete [ 2256.417156] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2261.995643] Lustre: server umount lustre-OST0000 complete [ 2265.177626] Lustre: server umount lustre-MDT0000 complete [ 2272.200967] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 2274.065499] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 2276.309192] vdc: vdc1 vdc9 [ 2278.756454] vde: vde1 vde9 [ 2281.155079] vdf: vdf1 vdf9 [ 2283.470108] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 2283.488502] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2283.522683] Lustre: lustre-MDT0000: new disk, initializing [ 2283.588332] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2284.401388] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2286.836320] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2289.882700] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2292.289114] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2293.023326] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2293.109404] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2293.137100] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2294.078558] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 2298.097145] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2300.623111] Lustre: server umount lustre-OST0000 complete [ 2301.698994] Lustre: server umount lustre-MDT0000 complete [ 2305.394714] Lustre: DEBUG MARKER: == conf-sanity test 50g: deactivated OST should not cause panic ========================================================== 08:44:47 (1715345087) [ 2306.732104] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2306.734548] Lustre: Skipped 6 previous similar messages [ 2307.442868] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2308.409755] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2311.032588] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2312.093308] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2321.698997] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2323.946016] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2325.329709] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 1 sec [ 2326.022429] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2329.055679] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2329.092057] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2329.132405] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 2329.862627] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b6cec000.ost_server_uuid 50 [ 2330.216012] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8800b6cec000.ost_server_uuid in FULL state after 0 sec [ 2330.497078] Lustre: Permanently deactivating lustre-OST0001 [ 2330.502592] Lustre: Setting parameter lustre-OST0001-osc.osc.active in log lustre-client [ 2331.104347] Lustre: Permanently reactivating lustre-OST0001 [ 2337.720961] Lustre: server umount lustre-OST0001 complete [ 2339.041081] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2339.047445] LustreError: Skipped 4 previous similar messages [ 2340.752258] Lustre: server umount lustre-OST0000 complete [ 2341.825606] Lustre: server umount lustre-MDT0000 complete [ 2348.276789] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 2349.555409] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 2351.665129] vdc: vdc1 vdc9 [ 2353.940674] vde: vde1 vde9 [ 2356.620494] vdf: vdf1 vdf9 [ 2359.092923] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 2359.094729] Lustre: Skipped 3 previous similar messages [ 2359.111095] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2359.112739] Lustre: Skipped 3 previous similar messages [ 2359.143921] Lustre: lustre-MDT0000: new disk, initializing [ 2359.145427] Lustre: Skipped 2 previous similar messages [ 2359.923519] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2362.173303] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2364.756226] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2364.783735] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2366.974439] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2367.688375] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2367.727570] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2369.729188] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2369.729339] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2369.729340] Lustre: Skipped 4 previous similar messages [ 2369.729727] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2369.729729] Lustre: Skipped 7 previous similar messages [ 2370.237790] Lustre: server umount lustre-OST0000 complete [ 2371.281974] Lustre: server umount lustre-MDT0000 complete [ 2374.597093] Lustre: DEBUG MARKER: == conf-sanity test 50h: LU-642: activate deactivated OST ========================================================== 08:45:56 (1715345156) [ 2377.266497] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2378.210133] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2379.395477] Lustre: Found index 0 for lustre-OST0000, updating log [ 2379.401633] Lustre: Permanently deactivating lustre-OST0000 [ 2379.404868] Lustre: Setting parameter lustre-OST0000-osc.osc.active in log lustre-client [ 2380.606209] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2381.587310] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2381.610736] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2384.541378] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2386.907183] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2387.118782] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2392.306771] Lustre: Permanently reactivating lustre-OST0000 [ 2400.576413] LustreError: 11-0: lustre-OST0001-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 2403.378580] Lustre: server umount lustre-OST0001 complete [ 2407.121247] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2410.477107] Lustre: server umount lustre-OST0000 complete [ 2411.733136] Lustre: server umount lustre-MDT0000 complete [ 2414.434058] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 2414.820071] Key type lgssc unregistered [ 2414.884828] LNet: 3223:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2415.887936] LNet: Removed LNI 192.168.203.126@tcp [ 2425.581461] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 2429.922576] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 2430.023282] alg: No test for adler32 (adler32-zlib) [ 2430.775655] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2430.824355] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 2430.865683] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 2430.867188] LNet: Accept secure, port 988 [ 2432.390662] Key type lgssc registered [ 2432.563837] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2434.490140] vdc: vdc1 vdc9 [ 2436.860712] vde: vde1 vde9 [ 2439.502299] vdf: vdf1 vdf9 [ 2442.700045] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2443.887396] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 2443.904972] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2443.946815] Lustre: lustre-MDT0000: new disk, initializing [ 2444.001990] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2444.017452] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2445.125087] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2447.935094] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2450.021400] Lustre: lustre-OST0000: new disk, initializing [ 2450.024110] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2450.026800] Lustre: Skipped 1 previous similar message [ 2450.055135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2451.264475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2451.269626] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2451.308101] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2451.781798] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2454.599071] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2456.273027] LustreError: 9109:0:(mdt_handler.c:3080:mdt_reint_internal()) cfs_fail_timeout id 142 sleeping for 10000ms [ 2460.370100] Lustre: lustre-OST0001: new disk, initializing [ 2460.373235] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2460.412647] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2462.183494] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2465.031137] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2465.622894] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 2466.279621] LustreError: 9109:0:(mdt_handler.c:3080:mdt_reint_internal()) cfs_fail_timeout id 142 awake [ 2466.290709] LustreError: 9109:0:(mdt_handler.c:3080:mdt_reint_internal()) cfs_fail_timeout id 142 sleeping for 10000ms [ 2468.916748] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2468.922846] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2468.965052] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2476.295565] LustreError: 9109:0:(mdt_handler.c:3080:mdt_reint_internal()) cfs_fail_timeout id 142 awake [ 2478.310658] Lustre: lustre-OST0001: Not available for connect from 192.168.203.26@tcp (stopping) [ 2478.897924] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2478.905697] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2482.842456] Lustre: server umount lustre-OST0001 complete [ 2483.310123] LustreError: 137-5: lustre-OST0001: not available for connect from 192.168.203.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2483.905622] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2488.913768] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2493.921735] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2496.321431] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2496.325751] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2496.332990] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2496.455548] Lustre: server umount lustre-OST0000 complete [ 2499.754853] Lustre: server umount lustre-MDT0000 complete [ 2502.857660] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 2503.567835] Key type lgssc unregistered [ 2503.656198] LNet: 15454:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2504.661026] LNet: Removed LNI 192.168.203.126@tcp [ 2512.925077] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 2517.162662] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 2517.348382] alg: No test for adler32 (adler32-zlib) [ 2518.099610] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2518.155238] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 2518.203430] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 2518.205769] LNet: Accept secure, port 988 [ 2519.739710] Key type lgssc registered [ 2520.069359] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2522.307663] vdc: vdc1 vdc9 [ 2524.716859] vde: vde1 vde9 [ 2527.476100] vdf: vdf1 vdf9 [ 2529.702462] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2530.853494] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 2530.874819] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2530.914947] Lustre: lustre-MDT0000: new disk, initializing [ 2530.973988] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2530.988170] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2531.737651] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2534.103691] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2535.693804] Lustre: lustre-OST0000: new disk, initializing [ 2535.696900] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2535.700010] Lustre: Skipped 1 previous similar message [ 2535.736866] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2537.263082] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2538.537665] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2538.543701] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2538.585016] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2539.590182] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2540.647387] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2540.684339] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2543.505189] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2543.508302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2547.302716] Lustre: server umount lustre-OST0000 complete [ 2548.435725] Lustre: server umount lustre-MDT0000 complete [ 2552.401449] Lustre: DEBUG MARKER: == conf-sanity test 52: check recovering objects from lost+found ========================================================== 08:48:54 (1715345334) [ 2552.824868] Lustre: DEBUG MARKER: SKIP: conf-sanity test_52 ldiskfs only test [ 2554.969735] Lustre: DEBUG MARKER: == conf-sanity test 53a: check OSS thread count params === 08:48:57 (1715345337) [ 2556.424932] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2557.219748] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2558.485924] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2559.824900] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2561.224657] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2562.531860] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2575.841879] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2575.841937] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2575.854274] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2576.720760] Lustre: server umount lustre-OST0000 complete [ 2578.010895] Lustre: server umount lustre-MDT0000 complete [ 2581.186947] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 2581.735103] Key type lgssc unregistered [ 2581.791007] LNet: 30018:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2582.793743] LNet: Removed LNI 192.168.203.126@tcp [ 2587.939736] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 2588.051450] alg: No test for adler32 (adler32-zlib) [ 2588.802627] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2588.852678] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 2588.894732] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 2588.896209] LNet: Accept secure, port 988 [ 2590.418656] Key type lgssc registered [ 2590.632938] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2592.180547] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2593.389551] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2594.132698] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2595.352412] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2596.760668] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2598.051656] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2599.124472] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2603.757505] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:35 to 0x240000400:65) [ 2608.769046] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2608.769058] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2608.769320] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2613.777084] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2614.067135] Lustre: server umount lustre-OST0000 complete [ 2615.113097] Lustre: server umount lustre-MDT0000 complete [ 2617.095111] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 2617.551698] Key type lgssc unregistered [ 2617.607811] LNet: 1737:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2618.609869] LNet: Removed LNI 192.168.203.126@tcp [ 2626.449699] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 2626.546032] alg: No test for adler32 (adler32-zlib) [ 2627.296621] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2627.345742] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 2627.387316] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 2627.388849] LNet: Accept secure, port 988 [ 2628.911617] Key type lgssc registered [ 2629.082009] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2630.302195] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2631.478471] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2632.230384] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2633.217948] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2634.445441] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2635.594889] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2636.764209] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2641.443232] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:67 to 0x240000400:97) [ 2651.457330] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2651.465059] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2654.463103] Lustre: server umount lustre-OST0000 complete [ 2655.545749] Lustre: server umount lustre-MDT0000 complete [ 2658.251830] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 2658.618556] Key type lgssc unregistered [ 2658.675849] LNet: 7368:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2659.677852] LNet: Removed LNI 192.168.203.126@tcp [ 2664.596625] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 2664.734063] alg: No test for adler32 (adler32-zlib) [ 2665.484638] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2665.532530] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 2665.571784] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 2665.573659] LNet: Accept secure, port 988 [ 2667.096742] Key type lgssc registered [ 2667.271765] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2668.458809] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2669.651166] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2670.412199] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2671.351260] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2672.732969] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2674.007470] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2675.037176] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2679.730954] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:99 to 0x240000400:129) [ 2684.737073] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2684.737078] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2684.737308] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2689.744998] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2690.001415] Lustre: server umount lustre-OST0000 complete [ 2691.143087] Lustre: server umount lustre-MDT0000 complete [ 2694.459217] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing unload_modules_local [ 2694.827488] Key type lgssc unregistered [ 2694.881860] LNet: 11620:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2695.883937] LNet: Removed LNI 192.168.203.126@tcp [ 2709.253454] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_hostid [ 2713.907868] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing load_modules_local [ 2714.045692] alg: No test for adler32 (adler32-zlib) [ 2714.796640] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2714.846299] Lustre: Lustre: Build Version: 2.15.63_2_ge8d3b4b [ 2714.886144] LNet: Added LNI 192.168.203.126@tcp [8/256/0/180] [ 2714.887393] LNet: Accept secure, port 988 [ 2716.409799] Key type lgssc registered [ 2716.597962] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2718.850541] vdc: vdc1 vdc9 [ 2721.333220] vde: vde1 vde9 [ 2723.915992] vdf: vdf1 vdf9 [ 2726.128207] vde: vde1 vde9 [ 2728.398182] vdf: vdf1 vdf9 [ 2730.633781] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2731.766695] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 2731.787835] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2731.821868] Lustre: lustre-MDT0000: new disk, initializing [ 2731.878947] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2731.892408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2732.710672] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2735.050270] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2736.584932] Lustre: lustre-OST2710: new disk, initializing [ 2736.586485] Lustre: srv-lustre-OST2710: No data found on store. Initialize space. [ 2736.588217] Lustre: Skipped 1 previous similar message [ 2736.607410] Lustre: lustre-OST2710: Imperative Recovery not enabled, recovery window 60-180 [ 2737.734462] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2739.914966] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST2710-osc-[-0-9a-f]*.ost_server_uuid [ 2741.362895] LustreError: 137-5: lustre-OST03e8: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2741.372073] Lustre: lustre-OST03e8: new disk, initializing [ 2741.373922] Lustre: srv-lustre-OST03e8: No data found on store. Initialize space. [ 2741.380627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:2710:ost [ 2741.382762] Lustre: cli-lustre-OST2710-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:2710:ost] [ 2741.396388] Lustre: lustre-OST03e8: Imperative Recovery not enabled, recovery window 60-180 [ 2741.409075] Lustre: lustre-OST2710-osc-MDT0000: update sequence from 0x127100000 to 0x240000400 [ 2742.695135] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing set_default_debug -1 all [ 2743.884393] Lustre: DEBUG MARKER: oleg326-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST03e8-osc-[-0-9a-f]*.ost_server_uuid [ 2746.004842] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST2710-osc-MDT0000.ost_server_uuid 50 [ 2746.065104] Lustre: DEBUG MARKER: os[cp].lustre-OST2710-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2746.446949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:3e8:ost [ 2746.452178] Lustre: cli-lustre-OST03e8-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:3e8:ost] [ 2746.483647] Lustre: lustre-OST03e8-osc-MDT0000: update sequence from 0x103e80000 to 0x280000400 [ 2746.879296] Lustre: DEBUG MARKER: oleg326-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST03e8-osc-MDT0000.ost_server_uuid 50 [ 2746.916354] Lustre: DEBUG MARKER: os[cp].lustre-OST03e8-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2747.337327] Lustre: DEBUG MARKER: conf-sanity test_56a: @@@@@@ FAIL: Stripe count not two: 1