[ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup subsys cpuacct [ 0.000000] Linux version 3.10.0-7.9-debug (green@centos7-base) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) ) #1 SMP Sat Mar 26 23:28:42 EDT 2022 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:centos7:ext4:ro:-p,-b4096 ro crashkernel=128M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000013edfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000000] e820: last_pfn = 0x13ee00 max_arch_pfn = 0x400000000 [ 0.000000] MTRR default type: write-back [ 0.000000] MTRR fixed ranges enabled: [ 0.000000] 00000-9FFFF write-back [ 0.000000] A0000-BFFFF uncachable [ 0.000000] C0000-FFFFF write-protect [ 0.000000] MTRR variable ranges enabled: [ 0.000000] 0 base 0000C0000000 mask 3FFFC0000000 uncachable [ 0.000000] 1 disabled [ 0.000000] 2 disabled [ 0.000000] 3 disabled [ 0.000000] 4 disabled [ 0.000000] 5 disabled [ 0.000000] 6 disabled [ 0.000000] 7 disabled [ 0.000000] PAT configuration [0-7]: WB WC UC- UC WB WP UC- UC [ 0.000000] e820: last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] mapped at [ffffffffff200b30] [ 0.000000] Base memory trampoline at [ffff880000099000] 99000 size 24576 [ 0.000000] Using GB pages for direct mapping [ 0.000000] BRK [0x023c1000, 0x023c1fff] PGTABLE [ 0.000000] BRK [0x023c2000, 0x023c2fff] PGTABLE [ 0.000000] BRK [0x023c3000, 0x023c3fff] PGTABLE [ 0.000000] BRK [0x023c4000, 0x023c4fff] PGTABLE [ 0.000000] BRK [0x023c5000, 0x023c5fff] PGTABLE [ 0.000000] BRK [0x023c6000, 0x023c6fff] PGTABLE [ 0.000000] RAMDISK: [mem 0xbc2e2000-0xbffbffff] [ 0.000000] Early table checksum verification disabled [ 0.000000] ACPI: RSDP 00000000000f5950 00014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 00000000bffe1bb7 00034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 00000000bffe1a53 00074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 00000000bffe0040 01A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 00000000bffe0000 00040 [ 0.000000] ACPI: APIC 00000000bffe1ac7 00090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 00000000bffe1b57 00038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 00000000bffe1b8f 00028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000013edfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x13e5e3000-0x13e609fff] [ 0.000000] Reserving 128MB of memory at 768MB for crashkernel (System RAM: 4077MB) [ 0.000000] kvm-clock: cpu 0, msr 1:3e592001, primary cpu clock [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 276740063 cycles [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x00001000-0x00ffffff] [ 0.000000] DMA32 [mem 0x01000000-0xffffffff] [ 0.000000] Normal [mem 0x100000000-0x13edfffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x00001000-0x0009efff] [ 0.000000] node 0: [mem 0x00100000-0xbffcdfff] [ 0.000000] node 0: [mem 0x100000000-0x13edfffff] [ 0.000000] Initmem setup node 0 [mem 0x00001000-0x13edfffff] [ 0.000000] On node 0 totalpages: 1043820 [ 0.000000] DMA zone: 64 pages used for memmap [ 0.000000] DMA zone: 21 pages reserved [ 0.000000] DMA zone: 3998 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 12224 pages used for memmap [ 0.000000] DMA32 zone: 782286 pages, LIFO batch:31 [ 0.000000] Normal zone: 4024 pages used for memmap [ 0.000000] Normal zone: 257536 pages, LIFO batch:31 [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] ACPI: IRQ0 used by override. [ 0.000000] ACPI: IRQ5 used by override. [ 0.000000] ACPI: IRQ9 used by override. [ 0.000000] ACPI: IRQ10 used by override. [ 0.000000] ACPI: IRQ11 used by override. [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] e820: [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 38 pages/cpu s115176 r8192 d32280 u524288 [ 0.000000] pcpu-alloc: s115176 r8192 d32280 u524288 alloc=1*2097152 [ 0.000000] pcpu-alloc: [0] 0 1 2 3 [ 0.000000] KVM setup async PF for cpu 0 [ 0.000000] kvm-stealtime: cpu 0, msr 13e2135c0 [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes) [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 1027487 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:centos7:ext4:ro:-p,-b4096 ro crashkernel=128M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] audit: disabled (until reboot) [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] x86/fpu: xstate_offset[2]: 0240, xstate_sizes[2]: 0100 [ 0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form [ 0.000000] Memory: 3820268k/5224448k available (8172k kernel code, 1049168k absent, 355012k reserved, 5773k data, 2532k init) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=4. [ 0.000000] Offload RCU callbacks from all CPUs [ 0.000000] Offload RCU callbacks from CPUs: 0-3. [ 0.000000] NR_IRQS:327936 nr_irqs:456 0 [ 0.000000] Console: colour *CGA 80x25 [ 0.000000] console [ttyS1] enabled [ 0.000000] allocated 25165824 bytes of page_cgroup [ 0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ODEBUG: 73 of 73 active objects replaced [ 0.000000] hpet clockevent registered [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.455584] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.458355] pid_max: default: 32768 minimum: 301 [ 0.459810] Security Framework initialized [ 0.461147] SELinux: Initializing. [ 0.462304] SELinux: Starting in permissive mode [ 0.463746] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) [ 0.468004] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) [ 0.470852] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes) [ 0.472816] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes) [ 0.475251] Initializing cgroup subsys memory [ 0.476540] Initializing cgroup subsys devices [ 0.477708] Initializing cgroup subsys freezer [ 0.478988] Initializing cgroup subsys net_cls [ 0.480285] Initializing cgroup subsys blkio [ 0.481499] Initializing cgroup subsys perf_event [ 0.482765] Initializing cgroup subsys hugetlb [ 0.483994] Initializing cgroup subsys pids [ 0.485194] Initializing cgroup subsys net_prio [ 0.486680] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.489448] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.490927] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.492383] tlb_flushall_shift: 6 [ 0.493422] FEATURE SPEC_CTRL Present [ 0.494433] FEATURE IBPB_SUPPORT Present [ 0.495561] Spectre V2 : Enabling Indirect Branch Prediction Barrier [ 0.497280] Spectre V2 : Vulnerable [ 0.498252] Speculative Store Bypass: Vulnerable [ 0.500358] debug: unmapping init [mem 0xffffffff82019000-0xffffffff8201ffff] [ 0.507644] ACPI: Core revision 20130517 [ 0.510515] ACPI: All ACPI Tables successfully acquired [ 0.512224] ftrace: allocating 30294 entries in 119 pages [ 0.558009] Enabling x2apic [ 0.558754] Enabled x2apic [ 0.559567] Switched APIC routing to physical x2apic. [ 0.562192] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.564082] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (fam: 06, model: 3e, stepping: 04) [ 0.566653] TSC deadline timer enabled [ 0.566830] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.568770] ... version: 2 [ 0.569634] ... bit width: 48 [ 0.570464] ... generic registers: 4 [ 0.571319] ... value mask: 0000ffffffffffff [ 0.572713] ... max period: 00007fffffffffff [ 0.574063] ... fixed-purpose events: 3 [ 0.575020] ... event mask: 000000070000000f [ 0.576399] KVM setup paravirtual spinlock [ 0.580764] kvm-clock: cpu 1, msr 1:3e592041, secondary cpu clock [ 0.584388] KVM setup async PF for cpu 1 [ 0.585818] kvm-stealtime: cpu 1, msr 13e2935c0 [ 0.588186] kvm-clock: cpu 2, msr 1:3e592081, secondary cpu clock [ 0.590626] KVM setup async PF for cpu 2 [ 0.579245] smpboot: Booting Node 0, Processors #1 #2 #3 OK [ 0.591065] kvm-clock: cpu 3, msr 1:3e5920c1, secondary cpu clock [ 0.593578] kvm-stealtime: cpu 2, msr 13e3135c0 [ 0.594788] Brought up 4 CPUs [ 0.594809] KVM setup async PF for cpu 3 [ 0.594815] kvm-stealtime: cpu 3, msr 13e3935c0 [ 0.597469] smpboot: Max logical packages: 1 [ 0.598427] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.601804] devtmpfs: initialized [ 0.603321] x86/mm: Memory block size: 128MB [ 0.608156] EVM: security.selinux [ 0.609148] EVM: security.ima [ 0.610113] EVM: security.capability [ 0.613275] atomic64 test passed for x86-64 platform with CX8 and with SSE [ 0.615731] NET: Registered protocol family 16 [ 0.617496] cpuidle: using governor haltpoll [ 0.619390] ACPI: bus type PCI registered [ 0.620556] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.622658] PCI: Using configuration type 1 for base access [ 0.624555] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.633098] ACPI: Added _OSI(Module Device) [ 0.634221] ACPI: Added _OSI(Processor Device) [ 0.635510] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.637044] ACPI: Added _OSI(Processor Aggregator Device) [ 0.638650] ACPI: Added _OSI(Linux-Dell-Video) [ 0.640610] ACPI: EC: Look up EC in DSDT [ 0.643644] ACPI: Interpreter enabled [ 0.644873] ACPI: (supports S0 S3 S4 S5) [ 0.645962] ACPI: Using IOAPIC for interrupt routing [ 0.647523] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.650617] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.658166] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.659992] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI] [ 0.662100] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM [ 0.664118] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.667956] acpiphp: Slot [2] registered [ 0.669041] acpiphp: Slot [3] registered [ 0.670135] acpiphp: Slot [4] registered [ 0.671334] acpiphp: Slot [5] registered [ 0.672634] acpiphp: Slot [6] registered [ 0.673775] acpiphp: Slot [7] registered [ 0.674569] acpiphp: Slot [8] registered [ 0.675639] acpiphp: Slot [9] registered [ 0.676687] acpiphp: Slot [10] registered [ 0.677949] acpiphp: Slot [11] registered [ 0.679495] acpiphp: Slot [12] registered [ 0.681058] acpiphp: Slot [13] registered [ 0.681837] acpiphp: Slot [14] registered [ 0.682890] acpiphp: Slot [15] registered [ 0.684249] acpiphp: Slot [16] registered [ 0.685493] acpiphp: Slot [17] registered [ 0.686634] acpiphp: Slot [18] registered [ 0.687455] acpiphp: Slot [19] registered [ 0.688437] acpiphp: Slot [20] registered [ 0.689492] acpiphp: Slot [21] registered [ 0.690244] acpiphp: Slot [22] registered [ 0.690953] acpiphp: Slot [23] registered [ 0.691935] acpiphp: Slot [24] registered [ 0.693239] acpiphp: Slot [25] registered [ 0.694209] acpiphp: Slot [26] registered [ 0.695072] acpiphp: Slot [27] registered [ 0.695874] acpiphp: Slot [28] registered [ 0.696638] acpiphp: Slot [29] registered [ 0.697582] acpiphp: Slot [30] registered [ 0.698741] acpiphp: Slot [31] registered [ 0.699860] PCI host bridge to bus 0000:00 [ 0.701009] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.702841] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.704544] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.706360] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.708470] pci_bus 0000:00: root bus resource [mem 0x140000000-0x1bfffffff window] [ 0.710735] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.712006] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.712591] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.713160] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.718466] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.720277] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.722105] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.724622] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.725803] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.727235] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.727601] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.728855] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.730347] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.731571] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.738000] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.738979] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.739720] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.746341] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.751609] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.760326] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.763682] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.767095] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.772639] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.786632] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.790934] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.794389] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.798640] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.811515] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 0.815948] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.820620] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.824927] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.838563] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 0.842347] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.844896] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.848270] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.854641] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 0.861139] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.862771] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.864357] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.875278] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 0.880388] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11) [ 0.882518] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11) [ 0.885077] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11) [ 0.886976] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11) [ 0.888887] ACPI: PCI Interrupt Link [LNKS] (IRQs *9) [ 0.891772] vgaarb: loaded [ 0.892660] SCSI subsystem initialized [ 0.893864] ACPI: bus type USB registered [ 0.895058] usbcore: registered new interface driver usbfs [ 0.896473] usbcore: registered new interface driver hub [ 0.897577] usbcore: registered new device driver usb [ 0.899248] PCI: Using ACPI for IRQ routing [ 0.900304] PCI: pci_cache_line_size set to 64 bytes [ 0.900561] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] [ 0.900566] e820: reserve RAM buffer [mem 0xbffce000-0xbfffffff] [ 0.900568] e820: reserve RAM buffer [mem 0x13ee00000-0x13fffffff] [ 0.900862] NetLabel: Initializing [ 0.901812] NetLabel: domain hash size = 128 [ 0.902989] NetLabel: protocols = UNLABELED CIPSOv4 [ 0.904377] NetLabel: unlabeled traffic allowed by default [ 0.906221] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.907654] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.912502] amd_nb: Cannot enumerate AMD northbridges [ 0.914249] Switched to clocksource kvm-clock [ 0.931616] pnp: PnP ACPI init [ 0.932612] ACPI: bus type PNP registered [ 0.933946] pnp 00:00: Plug and Play ACPI device, IDs PNP0303 (active) [ 0.934025] pnp 00:01: Plug and Play ACPI device, IDs PNP0f13 (active) [ 0.934066] pnp 00:02: [dma 2] [ 0.934103] pnp 00:02: Plug and Play ACPI device, IDs PNP0700 (active) [ 0.934177] pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active) [ 0.934259] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active) [ 0.934336] pnp 00:05: Plug and Play ACPI device, IDs PNP0b00 (active) [ 0.934683] pnp: PnP ACPI: found 6 devices [ 0.936051] ACPI: bus type PNP unregistered [ 0.946175] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.946182] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.946185] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.946188] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.946191] pci_bus 0000:00: resource 8 [mem 0x140000000-0x1bfffffff window] [ 0.946278] NET: Registered protocol family 2 [ 0.948304] TCP established hash table entries: 32768 (order: 6, 262144 bytes) [ 0.950856] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes) [ 0.953638] TCP: Hash tables configured (established 32768 bind 32768) [ 0.955765] TCP: reno registered [ 0.956505] UDP hash table entries: 2048 (order: 5, 196608 bytes) [ 0.957779] UDP-Lite hash table entries: 2048 (order: 5, 196608 bytes) [ 0.959323] NET: Registered protocol family 1 [ 0.960717] RPC: Registered named UNIX socket transport module. [ 0.961672] RPC: Registered udp transport module. [ 0.962435] RPC: Registered tcp transport module. [ 0.963233] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.964976] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.966529] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.967811] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.969388] PCI: CLS 0 bytes, default 64 [ 0.969545] Unpacking initramfs... [ 2.280920] debug: unmapping init [mem 0xffff8800bc2e2000-0xffff8800bffbffff] [ 2.283758] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.285099] software IO TLB [mem 0xb82e2000-0xbc2e2000] (64MB) mapped at [ffff8800b82e2000-ffff8800bc2e1fff] [ 2.287364] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 10737418240 ms ovfl timer [ 2.289380] RAPL PMU: hw unit of domain pp0-core 2^-0 Joules [ 2.290532] RAPL PMU: hw unit of domain package 2^-0 Joules [ 2.291389] RAPL PMU: hw unit of domain dram 2^-0 Joules [ 2.294196] cryptomgr_test (51) used greatest stack depth: 14128 bytes left [ 2.294650] futex hash table entries: 1024 (order: 4, 65536 bytes) [ 2.294725] Initialise system trusted keyring [ 2.322590] HugeTLB registered 1 GB page size, pre-allocated 0 pages [ 2.324664] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 2.329127] zpool: loaded [ 2.329673] zbud: loaded [ 2.330355] VFS: Disk quotas dquot_6.6.0 [ 2.331029] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 2.332634] NFS: Registering the id_resolver key type [ 2.333492] Key type id_resolver registered [ 2.334131] Key type id_legacy registered [ 2.334748] nfs4filelayout_init: NFSv4 File Layout Driver Registering... [ 2.336177] Key type big_key registered [ 2.336896] SELinux: Registering netfilter hooks [ 2.337921] cryptomgr_test (57) used greatest stack depth: 13968 bytes left [ 2.339879] cryptomgr_test (60) used greatest stack depth: 13664 bytes left [ 2.340278] NET: Registered protocol family 38 [ 2.340287] Key type asymmetric registered [ 2.340290] Asymmetric key parser 'x509' registered [ 2.340372] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250) [ 2.340469] io scheduler noop registered [ 2.340473] io scheduler deadline registered (default) [ 2.340538] io scheduler cfq registered [ 2.340543] io scheduler mq-deadline registered [ 2.340547] io scheduler kyber registered [ 2.342990] pci_hotplug: PCI Hot Plug PCI Core version: 0.5 [ 2.342996] pciehp: PCI Express Hot Plug Controller Driver version: 0.4 [ 2.351874] intel_idle: does not run on family 6 model 62 [ 2.351997] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.353130] ACPI: Power Button [PWRF] [ 2.353998] GHES: HEST is not enabled! [ 2.400567] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10 [ 2.460585] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 11 [ 2.566275] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11 [ 2.614380] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 10 [ 2.714297] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.741802] 00:03: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A [ 2.769478] 00:04: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A [ 2.771532] Non-volatile memory driver v1.3 [ 2.773022] Linux agpgart interface v0.103 [ 2.774729] crash memory driver: version 1.1 [ 2.776346] nbd: registered device at major 43 [ 2.780592] virtio-pci 0000:00:05.0: irq 24 for MSI/MSI-X [ 2.780639] virtio-pci 0000:00:05.0: irq 25 for MSI/MSI-X [ 2.780687] virtio-pci 0000:00:05.0: irq 26 for MSI/MSI-X [ 2.780721] virtio-pci 0000:00:05.0: irq 27 for MSI/MSI-X [ 2.780744] virtio-pci 0000:00:05.0: irq 28 for MSI/MSI-X [ 2.787102] virtio_blk virtio1: [vda] 66840 512-byte logical blocks (34.2 MB/32.6 MiB) [ 2.791001] virtio-pci 0000:00:06.0: irq 29 for MSI/MSI-X [ 2.791027] virtio-pci 0000:00:06.0: irq 30 for MSI/MSI-X [ 2.791049] virtio-pci 0000:00:06.0: irq 31 for MSI/MSI-X [ 2.791077] virtio-pci 0000:00:06.0: irq 32 for MSI/MSI-X [ 2.791104] virtio-pci 0000:00:06.0: irq 33 for MSI/MSI-X [ 2.796949] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.800593] virtio-pci 0000:00:07.0: irq 34 for MSI/MSI-X [ 2.800637] virtio-pci 0000:00:07.0: irq 35 for MSI/MSI-X [ 2.800671] virtio-pci 0000:00:07.0: irq 36 for MSI/MSI-X [ 2.800710] virtio-pci 0000:00:07.0: irq 37 for MSI/MSI-X [ 2.800732] virtio-pci 0000:00:07.0: irq 38 for MSI/MSI-X [ 2.805713] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.809537] virtio-pci 0000:00:08.0: irq 39 for MSI/MSI-X [ 2.809560] virtio-pci 0000:00:08.0: irq 40 for MSI/MSI-X [ 2.809587] virtio-pci 0000:00:08.0: irq 41 for MSI/MSI-X [ 2.809608] virtio-pci 0000:00:08.0: irq 42 for MSI/MSI-X [ 2.809629] virtio-pci 0000:00:08.0: irq 43 for MSI/MSI-X [ 2.814995] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.821086] virtio-pci 0000:00:09.0: irq 44 for MSI/MSI-X [ 2.821118] virtio-pci 0000:00:09.0: irq 45 for MSI/MSI-X [ 2.821141] virtio-pci 0000:00:09.0: irq 46 for MSI/MSI-X [ 2.821163] virtio-pci 0000:00:09.0: irq 47 for MSI/MSI-X [ 2.821185] virtio-pci 0000:00:09.0: irq 48 for MSI/MSI-X [ 2.828556] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.833441] virtio-pci 0000:00:0a.0: irq 49 for MSI/MSI-X [ 2.833471] virtio-pci 0000:00:0a.0: irq 50 for MSI/MSI-X [ 2.833493] virtio-pci 0000:00:0a.0: irq 51 for MSI/MSI-X [ 2.833514] virtio-pci 0000:00:0a.0: irq 52 for MSI/MSI-X [ 2.833534] virtio-pci 0000:00:0a.0: irq 53 for MSI/MSI-X [ 2.838272] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.842182] rdac: device handler registered [ 2.842956] hp_sw: device handler registered [ 2.843637] emc: device handler registered [ 2.844335] libphy: Fixed MDIO Bus: probed [ 2.846286] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 2.847450] ehci-pci: EHCI PCI platform driver [ 2.848285] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 2.849217] ohci-pci: OHCI PCI platform driver [ 2.849906] uhci_hcd: USB Universal Host Controller Interface driver [ 2.850980] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.853457] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.854261] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.855580] mousedev: PS/2 mouse device common for all mice [ 2.857119] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.859109] rtc_cmos 00:05: RTC can wake from S4 [ 2.861997] rtc_cmos 00:05: rtc core: registered rtc_cmos as rtc0 [ 2.863583] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.865878] hidraw: raw HID events driver (C) Jiri Kosina [ 2.867747] usbcore: registered new interface driver usbhid [ 2.868612] usbhid: USB HID core driver [ 2.869431] drop_monitor: Initializing network drop monitor service [ 2.870550] Netfilter messages via NETLINK v0.30. [ 2.871339] TCP: cubic registered [ 2.872095] Initializing XFRM netlink socket [ 2.873276] NET: Registered protocol family 10 [ 2.874542] NET: Registered protocol family 17 [ 2.875484] Key type dns_resolver registered [ 2.876406] mce: Using 10 MCE banks [ 2.877216] Loading compiled-in X.509 certificates [ 2.878782] Loaded X.509 cert 'Magrathea: Glacier signing key: e34d0e1b7fcf5b414cce75d36d8482945c781ed6' [ 2.880422] registered taskstats version 1 [ 2.883334] modprobe (71) used greatest stack depth: 13456 bytes left [ 2.886048] modprobe (73) used greatest stack depth: 13376 bytes left [ 2.887809] Key type trusted registered [ 2.892364] Key type encrypted registered [ 2.893177] IMA: No TPM chip found, activating TPM-bypass! (rc=-19) [ 2.896207] BERT: Boot Error Record Table support is disabled. Enable it by using bert_enable as kernel parameter. [ 2.898256] rtc_cmos 00:05: setting system clock to 2024-03-22 07:08:24 UTC (1711091304) [ 2.900077] debug: unmapping init [mem 0xffffffff81da0000-0xffffffff82018fff] [ 2.901463] Write protecting the kernel read-only data: 12288k [ 2.902694] debug: unmapping init [mem 0xffff8800017fe000-0xffff8800017fffff] [ 2.904323] debug: unmapping init [mem 0xffff880001b9b000-0xffff880001bfffff] [ 2.910350] random: systemd: uninitialized urandom read (16 bytes read) [ 2.912925] random: systemd: uninitialized urandom read (16 bytes read) [ 2.915020] random: systemd: uninitialized urandom read (16 bytes read) [ 2.917958] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN) [ 2.921834] systemd[1]: Detected virtualization kvm. [ 2.922920] systemd[1]: Detected architecture x86-64. [ 2.924479] systemd[1]: Running in initial RAM disk. [ 2.927652] systemd[1]: No hostname configured. [ 2.928894] systemd[1]: Set hostname to . [ 2.930433] random: systemd: uninitialized urandom read (16 bytes read) [ 2.932406] systemd[1]: Initializing machine ID from random generator. [ 2.967117] dracut-rootfs-g (86) used greatest stack depth: 13264 bytes left [ 2.969123] random: systemd: uninitialized urandom read (16 bytes read) [ 2.970361] random: systemd: uninitialized urandom read (16 bytes read) [ 2.971438] random: systemd: uninitialized urandom read (16 bytes read) [ 2.972698] random: systemd: uninitialized urandom read (16 bytes read) [ 2.974757] random: systemd: uninitialized urandom read (16 bytes read) [ 2.976059] random: systemd: uninitialized urandom read (16 bytes read) [ 2.982712] systemd[1]: Reached target Timers. [ 2.984791] systemd[1]: Reached target Local File Systems. [ 2.987355] systemd[1]: Created slice Root Slice. [ 2.989743] systemd[1]: Listening on Journal Socket. [ 2.992092] systemd[1]: Listening on udev Kernel Socket. [ 2.995069] systemd[1]: Created slice System Slice. [ 2.999018] systemd[1]: Starting Setup Virtual Console... [ 3.003671] systemd[1]: Starting Load Kernel Modules... [ 3.005503] systemd[1]: Reached target Slices. [ 3.008555] systemd[1]: Listening on udev Control Socket. [ 3.011835] systemd[1]: Reached target Sockets. [ 3.015626] systemd[1]: Starting Create list of required static device nodes for the current kernel... [ 3.021035] systemd[1]: Starting dracut cmdline hook... [ 3.024778] systemd[1]: Starting Journal Service... [ 3.027296] systemd[1]: Reached target Swap. [ 3.031055] systemd[1]: Started Setup Virtual Console. [ 3.036652] systemd[1]: Started Load Kernel Modules. [ 3.040108] systemd[1]: Started Create list of required static device nodes for the current kernel. [ 3.046839] systemd[1]: Starting Create Static Device Nodes in /dev... [ 3.051576] systemd[1]: Starting Apply Kernel Variables... [ 3.055899] systemd[1]: Started Journal Service. [ 3.063145] random: fast init done [ 3.294340] tsc: Refined TSC clocksource calibration: 2399.978 MHz [ 3.426152] libata version 3.00 loaded. [ 3.429198] ata_piix 0000:00:01.1: version 2.13 [ 3.457934] scsi host0: ata_piix [ 3.460111] scsi host1: ata_piix [ 3.462201] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 3.463615] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 3.505180] ip (316) used greatest stack depth: 13080 bytes left [ 3.546219] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready [ 3.548635] ip (343) used greatest stack depth: 12464 bytes left [ 3.693647] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input2 [ 3.846380] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [ 6.574135] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ 7.002044] systemd-journald[107]: Received SIGTERM from PID 1 (systemd). [ 7.236725] SELinux: Disabled at runtime. [ 7.238040] SELinux: Unregistering netfilter hooks [ 7.295994] ip_tables: (C) 2000-2006 Netfilter Core Team [ 7.298755] systemd[1]: Inserted module 'ip_tables' [ 7.735912] systemd-journald[564]: Received request to flush runtime journal from PID 1 [ 7.879363] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 7.916083] cryptd: max_cpu_qlen set to 1000 [ 7.938332] input: PC Speaker as /devices/platform/pcspkr/input/input3 [ 7.950796] AVX version of gcm_enc/dec engaged. [ 7.952034] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ 7.954971] AES CTR mode by8 optimization enabled [ 8.005601] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni) [ 8.013553] alg: No test for __generic-gcm-aes-aesni (__driver-generic-gcm-aes-aesni) [ 8.013757] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 8.128783] EDAC MC: Ver: 3.0.0 [ 8.137630] EDAC sbridge: Seeking for: PCI ID 8086:0ea0 [ 8.137654] EDAC sbridge: Ver: 1.1.2 [ 10.010706] mount.nfs (770) used greatest stack depth: 10512 bytes left [ 20.234369] libcfs: loading out-of-tree module taints kernel. [ 20.236021] libcfs: module verification failed: signature and/or required key missing - tainting kernel [ 25.158449] alg: No test for adler32 (adler32-zlib) [ 25.911128] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 26.045645] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 26.216679] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 26.218179] LNet: Accept secure, port 988 [ 27.762403] Key type lgssc registered [ 28.075527] Lustre: Echo OBD driver; http://www.lustre.org/ [ 28.532198] icp: module license 'CDDL' taints kernel. [ 28.534072] Disabling lock debugging due to kernel taint [ 31.086328] ZFS: Loaded module v0.8.6-1, ZFS pool version 5000, ZFS filesystem version 5 [ 33.192634] vdc: vdc1 vdc9 [ 35.594271] vde: vde1 vde9 [ 38.147824] vdf: vdf1 vdf9 [ 41.901963] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 43.702234] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 44.815154] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 44.871859] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 [ 44.909213] Lustre: lustre-MDT0000: new disk, initializing [ 44.966860] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 44.980569] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 45.005087] random: crng init done [ 45.026891] mount.lustre (6371) used greatest stack depth: 10136 bytes left [ 45.744910] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 48.801085] Lustre: lustre-OST0000: new disk, initializing [ 48.803152] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 48.805270] Lustre: Skipped 1 previous similar message [ 48.822268] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 50.111973] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 53.222520] Lustre: lustre-OST0001: new disk, initializing [ 53.225707] Lustre: srv-lustre-OST0001: No data found on store. Initialize space: rc = -61 [ 53.245121] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 53.894845] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 53.897761] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 53.941458] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 53.943749] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 54.522998] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 59.280607] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 65.417899] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 71.022693] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing check_logdir /tmp/testlogs/ [ 71.846479] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing yml_node [ 72.841221] Lustre: DEBUG MARKER: Client: 2.15.61.139 [ 73.401313] Lustre: DEBUG MARKER: MDS: 2.15.61.139 [ 73.973656] Lustre: DEBUG MARKER: OSS: 2.15.61.139 [ 74.347353] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Mar 22 03:09:35 EDT 2024 [ 75.750824] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball 110 [ 76.090680] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 88.926601] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 88.926800] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 88.934581] Lustre: Skipped 1 previous similar message [ 92.983777] Lustre: server umount lustre-MDT0000 complete [ 94.016918] LustreError: 8025:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711091395 with bad export cookie 14625637076497452069 [ 94.017690] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 94.023092] LustreError: 8025:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 100.024363] Lustre: server umount lustre-OST0000 complete [ 107.107812] Lustre: server umount lustre-OST0001 complete [ 108.463936] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 109.839391] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 111.781110] vdc: vdc1 vdc9 [ 113.964758] vde: vde1 vde9 [ 116.245356] vdf: vdf1 vdf9 [ 118.508118] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 118.531331] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 [ 118.570569] Lustre: lustre-MDT0000: new disk, initializing [ 118.638368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 118.654826] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 118.658852] Lustre: Skipped 1 previous similar message [ 118.706663] mount.lustre (16990) used greatest stack depth: 9640 bytes left [ 119.447547] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 121.737735] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 123.211003] Lustre: lustre-OST0000: new disk, initializing [ 123.212686] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 123.214550] Lustre: Skipped 1 previous similar message [ 123.233462] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 124.486936] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 124.570242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 124.572612] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 124.574647] Lustre: Skipped 1 previous similar message [ 124.595134] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 126.731204] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 127.502317] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 127.550306] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 129.551512] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 129.559525] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 129.562804] Lustre: Skipped 1 previous similar message [ 130.170602] Lustre: server umount lustre-OST0000 complete [ 131.340401] Lustre: server umount lustre-MDT0000 complete [ 133.587987] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 03:10:34 (1711091434) [ 134.960547] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 135.704092] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 136.714412] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 138.077709] mount.lustre (22636) used greatest stack depth: 9544 bytes left [ 139.258720] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 140.277960] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 147.069099] Lustre: Modifying parameter general.osc.*.max_dirty_mb in log params [ 150.063076] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 150.068591] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 151.893283] Lustre: server umount lustre-MDT0000 complete [ 152.900130] LustreError: 23552:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711091454 with bad export cookie 14625637076497453399 [ 152.901031] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 152.906851] LustreError: 23552:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 154.954054] Lustre: server umount lustre-OST0000 complete [ 158.926579] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 160.799854] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 160.802544] Lustre: Skipped 1 previous similar message [ 161.578688] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 164.638421] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 166.497108] Lustre: lustre-OST0001: new disk, initializing [ 166.499871] Lustre: srv-lustre-OST0001: No data found on store. Initialize space: rc = -61 [ 167.745135] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 168.009882] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 168.013512] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 168.045164] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 171.518321] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 172.363854] Lustre: Setting parameter general.lod.*.mdt_hash in log params [ 181.473691] Lustre: server umount lustre-MDT0000 complete [ 182.497892] LustreError: 25360:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711091484 with bad export cookie 14625637076497453917 [ 182.501068] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 192.497425] Lustre: 29898:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711091484/real 1711091484] req@ffff88008e96ad80 x1794209299340992/t0(0) o39->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711091494 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 192.518273] Lustre: server umount lustre-OST0000 complete [ 203.510423] Lustre: 30248:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711091495/real 1711091495] req@ffff88008e96ad80 x1794209299341440/t0(0) o39->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711091505 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 203.577649] Lustre: server umount lustre-OST0001 complete [ 206.750938] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 208.771346] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 208.773378] Lustre: Skipped 2 previous similar messages [ 209.555581] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 212.615806] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 215.714743] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 223.290721] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 224.192108] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 224.194339] Lustre: Skipped 1 previous similar message [ 232.480169] Lustre: server umount lustre-MDT0000 complete [ 233.578017] LustreError: 31524:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711091535 with bad export cookie 14625637076497454729 [ 233.583686] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 243.577383] Lustre: 2924:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711091535/real 1711091535] req@ffff88008e86fb80 x1794209299350592/t0(0) o39->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711091545 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 243.596378] Lustre: server umount lustre-OST0000 complete [ 254.652411] Lustre: 3262:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711091546/real 1711091546] req@ffff88008e86fb80 x1794209299351040/t0(0) o39->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711091556 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 254.724849] Lustre: server umount lustre-OST0001 complete [ 258.378838] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 03:12:39 (1711091559) [ 263.454086] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 266.150509] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 269.162172] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 272.190307] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 279.758897] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 280.683805] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 288.724896] Lustre: server umount lustre-MDT0000 complete [ 289.779038] LustreError: 7858:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711091591 with bad export cookie 14625637076497455408 [ 289.782288] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 299.778419] Lustre: 9525:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711091591/real 1711091591] req@ffff88009b838700 x1794209299360000/t0(0) o39->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711091601 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 299.798546] Lustre: server umount lustre-OST0000 complete [ 310.833397] Lustre: 9850:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711091602/real 1711091602] req@ffff88008e86df80 x1794209299360448/t0(0) o39->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711091612 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 310.902826] Lustre: server umount lustre-OST0001 complete [ 314.541345] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 03:13:35 (1711091615) [ 317.291680] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 319.241789] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 319.244026] Lustre: Skipped 5 previous similar messages [ 320.073419] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 323.299708] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 326.524469] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 334.067869] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 334.934910] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 342.147558] Lustre: lustre-MDT0000: Not available for connect from 192.168.203.16@tcp (stopping) [ 343.061467] Lustre: server umount lustre-MDT0000 complete [ 344.901166] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 345.012643] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 345.018947] Lustre: Skipped 1 previous similar message [ 345.867507] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 346.946520] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 348.333897] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 1 sec [ 350.046819] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 350.049144] Lustre: Skipped 1 previous similar message [ 355.341448] Lustre: server umount lustre-MDT0000 complete [ 356.453157] LustreError: 11805:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711091658 with bad export cookie 14625637076497456787 [ 356.457787] LustreError: 11805:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 356.464129] Lustre: server umount lustre-OST0000 complete [ 357.590289] Lustre: server umount lustre-OST0001 complete [ 361.332770] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 03:14:22 (1711091662) [ 364.179685] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 367.004272] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 370.192059] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 373.482093] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 381.124554] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 391.151018] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 391.151358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 391.151360] Lustre: Skipped 2 previous similar messages [ 391.162772] Lustre: Skipped 1 previous similar message [ 396.544686] Lustre: server umount lustre-MDT0000 complete [ 397.556940] LustreError: 19678:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711091699 with bad export cookie 14625637076497457221 [ 397.558066] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 397.558067] LustreError: Skipped 1 previous similar message [ 397.563904] LustreError: 19678:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 397.565730] Lustre: server umount lustre-OST0000 complete [ 398.590326] Lustre: server umount lustre-OST0001 complete [ 401.996362] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 03:15:03 (1711091703) [ 405.599902] Lustre: Setting parameter test1234-MDT0000.mdt.identity_upcall in log test1234-MDT0000 [ 405.602875] Lustre: Skipped 3 previous similar messages [ 405.630720] Lustre: ctl-test1234-MDT0000: No data found on store. Initialize space: rc = -61 [ 405.669786] Lustre: test1234-MDT0000: new disk, initializing [ 405.765796] Lustre: ctl-test1234-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 406.609741] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 413.217698] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 413.713207] Lustre: cli-test1234-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 413.736653] Lustre: test1234-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 418.687042] Lustre: test1234-MDT0000-lwp-OST0000: Connection to test1234-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 418.690702] Lustre: test1234-MDT0000: Not available for connect from 0@lo (stopping) [ 418.692200] Lustre: Skipped 3 previous similar messages [ 421.745855] Lustre: server umount test1234-MDT0000 complete [ 422.796540] Lustre: server umount test1234-OST0000 complete [ 426.261256] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 03:15:27 (1711091727) [ 426.620919] Lustre: DEBUG MARKER: SKIP: conf-sanity test_78 ldiskfs only test [ 428.453813] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 03:15:29 (1711091729) [ 429.997949] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 432.717296] vdc: vdc1 vdc9 [ 438.555226] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 439.930219] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 441.800840] vdc: vdc1 vdc9 [ 443.947134] vde: vde1 vde9 [ 446.219204] vdf: vdf1 vdf9 [ 448.433292] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 [ 448.434936] Lustre: Skipped 2 previous similar messages [ 448.466669] Lustre: lustre-MDT0000: new disk, initializing [ 448.467806] Lustre: Skipped 1 previous similar message [ 448.525912] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 448.527656] Lustre: Skipped 8 previous similar messages [ 448.540609] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 448.543291] Lustre: Skipped 1 previous similar message [ 449.283402] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 451.469559] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 454.090727] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 454.631335] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 454.653523] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 456.296138] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 457.051016] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 457.096637] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 459.631062] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 459.633329] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 459.636675] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 463.608518] Lustre: server umount lustre-OST0000 complete [ 464.915034] Lustre: server umount lustre-MDT0000 complete [ 468.511316] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 03:16:09 (1711091769) [ 470.564395] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 471.575602] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 474.066514] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 475.298387] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 476.365299] Lustre: 14085:0:(genops.c:1659:obd_export_evict_by_uuid()) MGS: evicting ac9f5470-b34d-48cb-92ea-97bf2c50ea66 at adminstrative request [ 477.919336] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 477.926008] LustreError: Skipped 1 previous similar message [ 477.929307] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 487.928783] Lustre: MGS: Client ac9f5470-b34d-48cb-92ea-97bf2c50ea66 (at 0@lo) reconnecting [ 487.930738] Lustre: Evicted from MGS (at 192.168.203.116@tcp) after server handle changed from 0xcaf8b36e5e69e82c to 0xcaf8b36e5e69e991 [ 487.935007] Lustre: MGC192.168.203.116@tcp: Connection restored to 192.168.203.116@tcp (at 0@lo) [ 509.085976] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 509.538415] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 509.542432] Lustre: Skipped 1 previous similar message [ 509.544609] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 509.573107] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 511.383552] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 514.526954] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 518.137068] Lustre: server umount lustre-OST0001 complete [ 519.534765] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 524.542725] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 524.543415] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 524.548598] Lustre: Skipped 2 previous similar messages [ 525.182056] Lustre: server umount lustre-OST0000 complete [ 526.263627] Lustre: server umount lustre-MDT0000 complete [ 529.604849] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 03:17:10 (1711091830) [ 529.955105] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 531.839477] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 03:17:13 (1711091833) [ 532.172735] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 534.011916] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 03:17:15 (1711091835) [ 534.351459] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 536.202750] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 03:17:17 (1711091837) [ 536.541652] Lustre: DEBUG MARKER: SKIP: conf-sanity test_83 ldiskfs only test [ 538.405221] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 03:17:19 (1711091839) [ 540.461407] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 541.503701] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 544.011989] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 545.045821] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 545.815813] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 549.802172] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 549.926074] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 552.577373] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 553.587892] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 554.376420] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 554.428644] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 561.046902] LustreError: 23993:0:(osd_handler.c:698:osd_ro()) lustre-MDT0000: *** setting device osd-zfs read-only *** [ 561.282031] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 566.159630] Lustre: Failing over lustre-MDT0000 [ 566.309042] Lustre: server umount lustre-MDT0000 complete [ 577.989639] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 578.989904] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 579.639356] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 579.639365] Lustre: 25260:0:(ldlm_lib.c:1992:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 579.674979] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 579.977355] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 580.384632] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 580.389484] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) Skipped 7 previous similar messages [ 580.392961] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 580.398578] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 1 previous similar message [ 580.702374] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 580.706070] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 1 previous similar message [ 581.513135] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 581.515847] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) Skipped 2 previous similar messages [ 581.517848] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 581.520018] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 2 previous similar messages [ 581.821373] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 581.823880] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 2 previous similar messages [ 583.832378] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 583.834394] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 4 previous similar messages [ 583.837170] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 583.842087] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) Skipped 5 previous similar messages [ 583.845921] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 583.848474] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 5 previous similar messages [ 586.152593] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to (at 0@lo) [ 588.059364] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 588.061696] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 10 previous similar messages [ 588.065523] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 588.068638] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) Skipped 10 previous similar messages [ 588.070617] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 588.072912] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 10 previous similar messages [ 596.296329] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 596.298773] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 21 previous similar messages [ 596.303155] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 596.306059] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) Skipped 21 previous similar messages [ 596.308428] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 596.310933] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 21 previous similar messages [ 612.361405] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 612.364176] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 42 previous similar messages [ 612.367832] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 612.371936] Lustre: 25371:0:(ldlm_lib.c:1992:extend_recovery_timer()) Skipped 42 previous similar messages [ 612.374201] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 612.377044] LustreError: 25371:0:(ldlm_lib.c:2625:replay_request_or_update()) Skipped 42 previous similar messages [ 639.653364] Lustre: 25371:0:(ldlm_lib.c:2300:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 639.658723] Lustre: 25371:0:(ldlm_lib.c:2310:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 639.663964] Lustre: 25371:0:(genops.c:1516:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client bafde894-8e6a-472b-bb2d-a24d78f69bc5@192.168.203.16@tcp [ 639.670995] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 639.674938] Lustre: 25371:0:(ldlm_lib.c:1824:abort_req_replay_queue()) @@@ aborted: req@ffff8800a8c9b480 x1794209295165376/t0(12884902056) o35->bafde894-8e6a-472b-bb2d-a24d78f69bc5@192.168.203.16@tcp:727/0 lens 392/0 e 0 to 0 dl 1711091957 ref 1 fl Complete:/204/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 [ 639.692882] Lustre: lustre-MDT0000: Denying connection for new client bafde894-8e6a-472b-bb2d-a24d78f69bc5 (at 192.168.203.16@tcp), waiting for 2 known clients (1 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 639.699015] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 639.721770] Lustre: lustre-MDT0000: Recovery over after 1:00, of 2 clients 1 recovered and 1 was evicted. [ 639.740211] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:107 to 0x240000400:609 [ 639.740218] Lustre: lustre-OST0001: deleting orphan objects from 0x280000400:75 to 0x280000400:577 [ 649.903023] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 649.905474] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 649.909212] Lustre: Skipped 3 previous similar messages [ 652.729613] Lustre: server umount lustre-OST0000 complete [ 654.927485] LustreError: 11-0: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 654.933961] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 654.937333] Lustre: Skipped 4 previous similar messages [ 655.901030] Lustre: server umount lustre-OST0001 complete [ 657.318343] Lustre: server umount lustre-MDT0000 complete [ 661.403671] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 03:19:22 (1711091962) [ 680.440675] LustreError: 28760:0:(mgc_request.c:251:do_config_log_add()) MGC192.168.203.116@tcp: failed processing log, type 1: rc = -5 [ 696.444040] LustreError: 28760:0:(mgc_request.c:251:do_config_log_add()) MGC192.168.203.116@tcp: failed processing log, type 4: rc = -110 [ 845.474618] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 845.479182] Lustre: Skipped 8 previous similar messages [ 847.265139] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 848.624493] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 849.281818] Lustre: server umount lustre-OST0000 complete [ 853.794968] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 03:22:34 (1711092154) [ 854.152082] Lustre: DEBUG MARKER: SKIP: conf-sanity test_86 LU-6442: no such mkfs params for ZFS OSTs [ 856.442004] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 03:22:37 (1711092157) [ 856.790452] Lustre: DEBUG MARKER: SKIP: conf-sanity test_87 ldiskfs only test [ 858.938507] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 03:22:40 (1711092160) [ 859.315594] Lustre: DEBUG MARKER: SKIP: conf-sanity test_88 LU-6662: no implementation for ZFS [ 861.464092] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 03:22:42 (1711092162) [ 875.267369] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 877.750592] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 879.802271] Lustre: server umount lustre-MDT0000 complete [ 890.901903] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 892.490014] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 894.873547] vdc: vdc1 vdc9 [ 898.058757] vde: vde1 vde9 [ 901.177567] vdf: vdf1 vdf9 [ 903.908376] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 903.912682] Lustre: Skipped 1 previous similar message [ 903.935651] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 [ 903.938080] Lustre: Skipped 3 previous similar messages [ 903.979073] Lustre: lustre-MDT0000: new disk, initializing [ 903.980765] Lustre: Skipped 2 previous similar messages [ 904.051206] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 905.115204] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 907.975508] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 911.834218] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 912.012636] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 912.058177] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 914.721328] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 915.874944] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 915.962185] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 916.975352] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 916.981807] Lustre: Skipped 1 previous similar message [ 916.984386] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 918.715384] Lustre: server umount lustre-OST0000 complete [ 920.326526] Lustre: server umount lustre-MDT0000 complete [ 925.318528] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 03:23:46 (1711092226) [ 927.558430] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 928.678357] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 931.623668] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 932.919638] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 940.045940] Lustre: *** cfs_fail_loc=159, val=0*** [ 956.060726] Lustre: lustre-MDT0000: Client d67690af-9f0f-45e5-bf29-f24185ed94a8 (at 192.168.203.16@tcp) reconnecting [ 956.331403] Lustre: *** cfs_fail_loc=159, val=0*** [ 956.332667] Lustre: Skipped 6 previous similar messages [ 972.332616] Lustre: lustre-MDT0000: Client d67690af-9f0f-45e5-bf29-f24185ed94a8 (at 192.168.203.16@tcp) reconnecting [ 975.189342] Lustre: server umount lustre-OST0000 complete [ 986.436436] Lustre: 14996:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711092278/real 1711092278] req@ffff8801317e5f80 x1794209299450368/t0(0) o9->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 1 dl 1711092288 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 986.589359] Lustre: server umount lustre-MDT0000 complete [ 989.508133] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 989.944992] Key type lgssc unregistered [ 990.003656] LNet: 15575:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 990.007097] LNet: Removed LNI 192.168.203.116@tcp [ 997.737941] alg: No test for adler32 (adler32-zlib) [ 998.490353] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 998.545212] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 998.587146] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 998.589026] LNet: Accept secure, port 988 [ 1000.116420] Key type lgssc registered [ 1000.305020] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1001.659025] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1002.888162] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1003.765212] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1004.959741] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1006.697414] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1007.977236] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1009.274931] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1012.691439] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:42 to 0x240000400:65 [ 1019.494825] Lustre: *** cfs_fail_loc=159, val=0*** [ 1035.515985] Lustre: lustre-MDT0000: Client 634c17b4-5a07-4a08-8e54-9a97adb4c10e (at 192.168.203.16@tcp) reconnecting [ 1036.084487] Lustre: *** cfs_fail_loc=159, val=0*** [ 1052.088537] Lustre: lustre-MDT0000: Client 634c17b4-5a07-4a08-8e54-9a97adb4c10e (at 192.168.203.16@tcp) reconnecting [ 1052.406817] Lustre: *** cfs_fail_loc=159, val=0*** [ 1052.409328] Lustre: Skipped 4 previous similar messages [ 1068.407881] Lustre: lustre-MDT0000: Client 634c17b4-5a07-4a08-8e54-9a97adb4c10e (at 192.168.203.16@tcp) reconnecting [ 1069.186774] Lustre: 19925:0:(mdt_handler.c:84:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1069.768928] Lustre: *** cfs_fail_loc=159, val=0*** [ 1069.771003] Lustre: Skipped 3 previous similar messages [ 1085.774643] Lustre: lustre-MDT0000: Client d4f26d07-e130-4dae-be9f-36616e9d01c2 (at 192.168.203.16@tcp) reconnecting [ 1086.141932] Lustre: *** cfs_fail_loc=159, val=0*** [ 1086.144816] Lustre: Skipped 14 previous similar messages [ 1102.146209] Lustre: lustre-MDT0000: Client d4f26d07-e130-4dae-be9f-36616e9d01c2 (at 192.168.203.16@tcp) reconnecting [ 1102.526689] Lustre: 20408:0:(mdt_handler.c:84:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1104.381741] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 1104.388026] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1104.398112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1105.784657] Lustre: server umount lustre-OST0000 complete [ 1106.854215] Lustre: server umount lustre-MDT0000 complete [ 1110.325178] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1111.008025] Key type lgssc unregistered [ 1111.096875] LNet: 21438:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1111.100701] LNet: Removed LNI 192.168.203.116@tcp [ 1120.867324] alg: No test for adler32 (adler32-zlib) [ 1121.618343] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1121.670027] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 1121.714190] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1121.716256] LNet: Accept secure, port 988 [ 1123.240558] Key type lgssc registered [ 1123.547645] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1124.794590] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1125.974306] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1126.668522] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1128.031092] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1129.617377] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1130.743426] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1131.997844] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1135.614846] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:91 to 0x240000400:129 [ 1140.174541] Lustre: 25086:0:(mdt_handler.c:84:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1140.784476] Lustre: 25210:0:(mdt_handler.c:84:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1141.313971] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight in log params [ 1142.485101] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight in log params [ 1142.488990] Lustre: Skipped 1 previous similar message [ 1143.159720] Lustre: 25571:0:(mdt_handler.c:84:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1145.631054] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1145.638906] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1146.008146] Lustre: server umount lustre-OST0000 complete [ 1147.393448] Lustre: server umount lustre-MDT0000 complete [ 1150.620184] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1151.017901] Key type lgssc unregistered [ 1151.075573] LNet: 26651:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1151.077783] LNet: Removed LNI 192.168.203.116@tcp [ 1160.603536] alg: No test for adler32 (adler32-zlib) [ 1161.354336] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1161.408125] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 1161.449561] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1161.451053] LNet: Accept secure, port 988 [ 1162.973413] Key type lgssc registered [ 1163.233001] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1165.191664] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1166.488019] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1167.494964] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1168.769143] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1170.405235] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1172.027193] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1173.496638] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1176.405053] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:91 to 0x240000400:161 [ 1181.276461] Lustre: *** cfs_fail_loc=159, val=0*** [ 1197.284073] Lustre: lustre-MDT0000: Client a2b0be97-fcc3-449b-82cd-af6fc779a4f5 (at 192.168.203.16@tcp) reconnecting [ 1198.651968] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 1198.656952] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1198.666500] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1201.438938] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1204.264208] Lustre: server umount lustre-OST0000 complete [ 1205.876407] Lustre: server umount lustre-MDT0000 complete [ 1208.793753] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1209.556734] Key type lgssc unregistered [ 1209.646764] LNet: 31608:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1209.651037] LNet: Removed LNI 192.168.203.116@tcp [ 1217.381574] alg: No test for adler32 (adler32-zlib) [ 1218.132363] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1218.183815] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 1218.226858] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1218.229312] LNet: Accept secure, port 988 [ 1219.754431] Key type lgssc registered [ 1219.921668] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1221.259469] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1222.434857] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1223.120175] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1224.031750] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1225.212436] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1226.335844] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1227.272610] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1233.229611] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:170 to 0x240000400:193 [ 1250.405294] Lustre: lustre-OST0000: evicting 6ceb0dac-2216-4c6c-b641-e8787c6d3862 (at 192.168.203.16@tcp) by administrative request [ 1278.287303] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1278.287473] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1278.287997] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1280.265622] Lustre: server umount lustre-OST0000 complete [ 1281.857406] Lustre: server umount lustre-MDT0000 complete [ 1285.584795] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1286.226143] Key type lgssc unregistered [ 1286.308889] LNet: 4754:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1286.312769] LNet: Removed LNI 192.168.203.116@tcp [ 1319.087648] alg: No test for adler32 (adler32-zlib) [ 1319.839392] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1319.901888] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 1319.955951] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1319.958612] LNet: Accept secure, port 988 [ 1321.489587] Key type lgssc registered [ 1321.801884] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1323.175932] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1324.359640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1325.059223] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1326.018127] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1327.224739] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1328.472123] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1329.739758] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1334.224224] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:195 to 0x240000400:225 [ 1339.231466] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1339.231534] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1339.231946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1344.239098] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1344.398367] Lustre: server umount lustre-OST0000 complete [ 1345.922458] Lustre: server umount lustre-MDT0000 complete [ 1348.751013] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1349.169192] Key type lgssc unregistered [ 1349.239508] LNet: 13281:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1349.241637] LNet: Removed LNI 192.168.203.116@tcp [ 1363.234392] alg: No test for adler32 (adler32-zlib) [ 1363.985326] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1364.042444] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 1364.094936] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1364.097097] LNet: Accept secure, port 988 [ 1365.632496] Key type lgssc registered [ 1365.956278] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1368.160264] vdc: vdc1 vdc9 [ 1371.009582] vde: vde1 vde9 [ 1374.545454] vdf: vdf1 vdf9 [ 1377.212645] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1378.386100] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1378.407298] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 [ 1378.440359] Lustre: lustre-MDT0000: new disk, initializing [ 1378.501007] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1378.514946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1379.525893] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1382.481204] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1384.524430] Lustre: lustre-OST0000: new disk, initializing [ 1384.527188] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 1384.530361] Lustre: Skipped 1 previous similar message [ 1384.549878] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1385.363431] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1385.366081] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1385.394564] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1385.908493] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1388.333227] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1393.470478] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 1393.472527] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1393.475766] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1395.359305] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1396.964144] Lustre: server umount lustre-OST0000 complete [ 1398.593743] Lustre: server umount lustre-MDT0000 complete [ 1402.312657] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 1403.052358] Key type lgssc unregistered [ 1403.138845] LNet: 23163:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1403.143564] LNet: Removed LNI 192.168.203.116@tcp [ 1413.221195] alg: No test for adler32 (adler32-zlib) [ 1413.972308] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1414.023821] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 1414.066553] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 1414.068036] LNet: Accept secure, port 988 [ 1415.592514] Key type lgssc registered [ 1415.918483] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1417.866881] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1419.173168] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1420.260715] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1421.252365] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1422.514726] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1423.785743] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 1424.844470] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1428.528422] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:3 to 0x240000400:33 [ 1433.222559] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1433.233883] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1433.241360] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1433.245738] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1433.251981] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to (at 0@lo) [ 1433.902363] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1433.907703] Lustre: Skipped 1 previous similar message [ 1433.915367] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1433.921866] Lustre: Skipped 1 previous similar message [ 1433.924178] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1433.928261] Lustre: Skipped 1 previous similar message [ 1433.930764] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1433.939754] LustreError: Skipped 1 previous similar message [ 1433.944621] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to (at 0@lo) [ 1433.948529] Lustre: Skipped 1 previous similar message [ 1434.962309] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1434.969408] Lustre: Skipped 2 previous similar messages [ 1434.979774] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1434.989289] Lustre: Skipped 2 previous similar messages [ 1434.992897] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1434.999141] Lustre: Skipped 2 previous similar messages [ 1435.003385] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1435.009670] LustreError: Skipped 2 previous similar messages [ 1435.012681] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to (at 0@lo) [ 1435.017597] Lustre: Skipped 2 previous similar messages [ 1437.046104] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1437.050809] Lustre: Skipped 5 previous similar messages [ 1437.061092] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1437.070198] Lustre: Skipped 5 previous similar messages [ 1437.073662] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1437.078686] Lustre: Skipped 5 previous similar messages [ 1437.082154] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1437.088863] LustreError: Skipped 5 previous similar messages [ 1437.091737] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to (at 0@lo) [ 1437.096626] Lustre: Skipped 5 previous similar messages [ 1440.386701] LustreError: 25009:0:(osp_precreate.c:670:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 1440.392409] LustreError: 25009:0:(osp_precreate.c:1374:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 1441.072184] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1441.078447] Lustre: Skipped 12 previous similar messages [ 1441.087992] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1441.094851] Lustre: Skipped 12 previous similar messages [ 1441.097529] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1441.103836] Lustre: Skipped 12 previous similar messages [ 1441.107678] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1441.113570] LustreError: Skipped 12 previous similar messages [ 1441.116390] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to (at 0@lo) [ 1441.119705] Lustre: Skipped 12 previous similar messages [ 1447.275429] LustreError: 25009:0:(osp_precreate.c:670:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 1447.280710] LustreError: 25009:0:(osp_precreate.c:1374:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 1449.153822] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1449.155948] Lustre: Skipped 29 previous similar messages [ 1449.159809] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1449.163846] Lustre: Skipped 29 previous similar messages [ 1449.165491] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1449.167752] Lustre: Skipped 29 previous similar messages [ 1449.169468] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1449.174964] LustreError: Skipped 29 previous similar messages [ 1449.176511] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to (at 0@lo) [ 1449.179593] Lustre: Skipped 29 previous similar messages [ 1465.351078] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1465.354166] Lustre: Skipped 62 previous similar messages [ 1465.360633] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1465.367335] Lustre: Skipped 62 previous similar messages [ 1465.369686] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1465.371897] Lustre: Skipped 62 previous similar messages [ 1465.373511] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1465.376460] LustreError: Skipped 62 previous similar messages [ 1465.378273] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to (at 0@lo) [ 1465.381368] Lustre: Skipped 62 previous similar messages [ 1497.431901] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1497.435494] Lustre: Skipped 135 previous similar messages [ 1497.442612] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1497.449119] Lustre: Skipped 135 previous similar messages [ 1497.451823] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1497.454433] Lustre: Skipped 135 previous similar messages [ 1497.455866] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1497.464690] LustreError: Skipped 135 previous similar messages [ 1497.468725] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to (at 0@lo) [ 1497.470824] Lustre: Skipped 135 previous similar messages [ 1500.023403] LustreError: 25009:0:(osp_precreate.c:670:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 1500.030961] LustreError: 25009:0:(osp_precreate.c:1374:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 1535.821117] LustreError: 25009:0:(osp_precreate.c:670:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 1535.823533] LustreError: 25009:0:(osp_precreate.c:1374:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 1561.612298] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1561.615993] Lustre: Skipped 230 previous similar messages [ 1561.623918] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1561.632095] Lustre: Skipped 230 previous similar messages [ 1561.635912] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1561.640111] Lustre: Skipped 230 previous similar messages [ 1561.642954] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1561.649932] LustreError: Skipped 230 previous similar messages [ 1561.652638] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to (at 0@lo) [ 1561.657174] Lustre: Skipped 230 previous similar messages [ 1575.961201] LustreError: 25009:0:(osp_precreate.c:670:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 1575.966464] LustreError: 25009:0:(osp_precreate.c:1374:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 1604.275115] LustreError: 25009:0:(osp_precreate.c:670:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 1604.282956] LustreError: 25009:0:(osp_precreate.c:1374:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 1677.128476] LustreError: 25009:0:(osp_precreate.c:670:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 1677.133112] LustreError: 25009:0:(osp_precreate.c:1374:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 1689.662650] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1689.665292] Lustre: Skipped 429 previous similar messages [ 1689.671179] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1689.674781] Lustre: Skipped 429 previous similar messages [ 1689.676294] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1689.678507] Lustre: Skipped 429 previous similar messages [ 1689.680140] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1689.685968] LustreError: Skipped 429 previous similar messages [ 1689.688807] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to (at 0@lo) [ 1689.692338] Lustre: Skipped 429 previous similar messages [ 2319.226569] Lustre: lustre-MDT0000: Client 5a8b222e-57f2-4967-866d-db012deba3a1 (at 192.168.203.16@tcp) reconnecting [ 2319.230367] Lustre: Skipped 85 previous similar messages [ 2330.225231] Lustre: 26014:0:(mdt_recovery.c:148:mdt_req_from_lrd()) @@@ restoring transno req@ffff88008985ad80 x1794210766443840/t8590066657(0) o36->5a8b222e-57f2-4967-866d-db012deba3a1@192.168.203.16@tcp:147/0 lens 504/2888 e 0 to 0 dl 1711093642 ref 1 fl Interpret:/202/0 rc 0/0 job:'' uid:0 gid:0 [ 2400.302208] Lustre: 29361:0:(mdt_recovery.c:148:mdt_req_from_lrd()) @@@ restoring transno req@ffff88008b05ea00 x1794210767055808/t8590071429(0) o36->5a8b222e-57f2-4967-866d-db012deba3a1@192.168.203.16@tcp:217/0 lens 504/2888 e 0 to 0 dl 1711093712 ref 1 fl Interpret:/202/0 rc 0/0 job:'' uid:0 gid:0 [ 2423.842937] Lustre: 29361:0:(mdt_recovery.c:148:mdt_req_from_lrd()) @@@ restoring transno req@ffff880130981c00 x1794210767236480/t8590072836(0) o36->5a8b222e-57f2-4967-866d-db012deba3a1@192.168.203.16@tcp:241/0 lens 504/2888 e 0 to 0 dl 1711093736 ref 1 fl Interpret:/202/0 rc 0/0 job:'' uid:0 gid:0 [ 2489.360313] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2489.363603] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2489.371337] Lustre: Skipped 85 previous similar messages [ 2489.374661] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2489.638777] Lustre: server umount lustre-OST0000 complete [ 2491.322619] Lustre: server umount lustre-MDT0000 complete [ 2495.466734] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 2496.458733] Key type lgssc unregistered [ 2496.571266] LNet: 14281:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2496.575184] LNet: Removed LNI 192.168.203.116@tcp [ 2510.288461] alg: No test for adler32 (adler32-zlib) [ 2511.041386] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2511.191419] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 2511.287921] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 2511.291930] LNet: Accept secure, port 988 [ 2512.857226] Key type lgssc registered [ 2513.255752] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2515.831466] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2517.456592] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2519.034896] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2521.410715] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2524.336688] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2527.061863] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2528.315924] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:50034 to 0x240000400:50049 [ 2529.568059] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2533.384022] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2534.745572] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2534.768480] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2534.772047] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.796619] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.203.116@tcp (at 0@lo) [ 2534.797491] LustreError: 16157:0:(osp_precreate.c:1282:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 2536.171969] Lustre: lustre-OST0000: Not available for connect from 192.168.203.16@tcp (stopping) [ 2538.320116] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2538.336734] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2541.175677] Lustre: lustre-OST0000: Not available for connect from 192.168.203.16@tcp (stopping) [ 2542.141335] Lustre: server umount lustre-OST0000 complete [ 2554.877710] LustreError: 16157:0:(osp_precreate.c:1282:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 2570.783972] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2573.113091] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2574.892534] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2577.770489] LustreError: 167-0: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.777493] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.203.116@tcp (at 0@lo) [ 2592.434672] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 16 sec [ 2593.695745] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012a6c0000.ost_server_uuid 50 [ 2594.439875] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff88012a6c0000.ost_server_uuid in FULL state after 0 sec [ 2597.813379] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2597.821728] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2601.937515] Lustre: server umount lustre-OST0000 complete [ 2604.087100] Lustre: server umount lustre-MDT0000 complete [ 2608.228991] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 2609.146885] Key type lgssc unregistered [ 2609.241140] LNet: 20946:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2609.248973] LNet: Removed LNI 192.168.203.116@tcp [ 2622.413640] alg: No test for adler32 (adler32-zlib) [ 2623.169478] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2623.293021] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 2623.392784] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 2623.395716] LNet: Accept secure, port 988 [ 2624.962665] Key type lgssc registered [ 2625.386040] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2628.451485] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2629.828031] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2631.458461] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2635.244565] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2637.388231] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2640.223697] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:50051 to 0x240000400:50081 [ 2641.992579] Lustre: lustre-OST0001: new disk, initializing [ 2641.995899] Lustre: srv-lustre-OST0001: No data found on store. Initialize space: rc = -61 [ 2642.061039] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2643.885399] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2643.888644] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2643.987108] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2644.875766] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2652.143648] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2656.102847] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 2671.935726] Lustre: server umount lustre-MDT0000 complete [ 2673.605701] LustreError: 23097:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711093975 with bad export cookie 16639292123303814059 [ 2673.614829] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2683.605232] Lustre: 27661:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711093975/real 1711093975] req@ffff880087bb8000 x1794212022524224/t0(0) o39->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711093985 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 2683.668136] Lustre: server umount lustre-OST0000 complete [ 2695.079138] Lustre: 27881:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711093986/real 1711093986] req@ffff8800a6405180 x1794212022524672/t0(0) o39->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711093996 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 2695.307042] Lustre: server umount lustre-OST0001 complete [ 2708.925302] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 2712.718387] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2714.026166] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2717.127025] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2719.470961] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2723.815257] Lustre: mylustre-OST0000: deleting orphan objects from 0x240000400:50084 to 0x240000400:50113 [ 2725.503483] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2730.109094] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2731.931161] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 2742.784382] Lustre: server umount mylustre-MDT0000 complete [ 2744.325747] LustreError: 30662:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711094045 with bad export cookie 16639292123303816236 [ 2744.329902] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2754.333398] Lustre: 1562:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711094045/real 1711094045] req@ffff8800a7062d80 x1794212022537344/t0(0) o39->mylustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711094055 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 2754.375196] Lustre: server umount mylustre-OST0000 complete [ 2765.963409] Lustre: 1791:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711094057/real 1711094057] req@ffff8800ad306d80 x1794212022537664/t0(0) o39->mylustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711094067 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 2766.114905] Lustre: server umount mylustre-OST0001 complete [ 2779.944914] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 2784.220988] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2784.223889] Lustre: Skipped 1 previous similar message [ 2786.139479] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2791.924810] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2793.663687] LustreError: 137-5: tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2793.671900] Lustre: tfs-OST0000: deleting orphan objects from 0x240000400:50115 to 0x240000400:50145 [ 2797.625870] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2802.054818] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2803.685721] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 2815.040415] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2815.041856] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 2815.049592] Lustre: Skipped 1 previous similar message [ 2817.172411] Lustre: server umount tfs-MDT0000 complete [ 2818.663786] LustreError: 7239:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711094120 with bad export cookie 16639292123303817447 [ 2818.666445] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2818.725216] LustreError: 7239:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2818.729590] Lustre: server umount tfs-OST0000 complete [ 2820.515180] Lustre: server umount tfs-OST0001 complete [ 2833.222815] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 2837.725163] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2837.731485] Lustre: Skipped 2 previous similar messages [ 2839.240854] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2846.229700] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2848.301966] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2848.312372] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:50115 to 0x240000400:50177 [ 2849.176199] hrtimer: interrupt took 5946677 ns [ 2853.081609] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2857.393459] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2859.146180] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 2872.508548] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 03:56:12 (1711094172) [ 2884.896991] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2884.901866] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2884.901873] Lustre: Skipped 1 previous similar message [ 2884.913169] Lustre: Skipped 1 previous similar message [ 2889.903578] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2889.912273] Lustre: Skipped 2 previous similar messages [ 2891.079387] Lustre: server umount lustre-MDT0000 complete [ 2893.750759] LustreError: 11030:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711094195 with bad export cookie 16639292123303818462 [ 2893.753435] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2893.773865] LustreError: 11030:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2899.794524] Lustre: server umount lustre-OST0000 complete [ 2908.401009] Lustre: server umount lustre-OST0001 complete [ 2911.570653] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 2915.186172] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 2920.186841] vdc: vdc1 vdc9 [ 2925.383394] vde: vde1 vde9 [ 2925.426692] vde: vde1 vde9 [ 2930.996679] vdf: vdf1 vdf9 [ 2935.958781] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 2936.018049] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 [ 2936.057570] Lustre: lustre-MDT0000: new disk, initializing [ 2936.278126] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2936.283341] Lustre: Skipped 2 previous similar messages [ 2936.338349] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2937.960439] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2943.707628] Lustre: lustre-OST0000: new disk, initializing [ 2943.712055] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 2943.715615] Lustre: Skipped 1 previous similar message [ 2946.606195] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2950.148024] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2950.151683] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2950.242487] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2952.410992] Lustre: lustre-OST0001: new disk, initializing [ 2952.423284] Lustre: srv-lustre-OST0001: No data found on store. Initialize space: rc = -61 [ 2954.187094] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2954.191236] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2954.296444] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2955.165233] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 2963.769675] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 03:57:44 (1711094264) [ 2972.602852] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 03:57:53 (1711094273) [ 2984.097285] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2984.107438] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2986.045700] Lustre: server umount lustre-OST0000 complete [ 2989.105015] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2989.111377] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2993.976229] Lustre: server umount lustre-MDT0000 complete [ 3000.135596] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3004.867170] LustreError: 18795:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711094306 with bad export cookie 16639292123303819365 [ 3004.867535] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3004.877649] LustreError: 18795:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3011.097383] Lustre: server umount lustre-OST0001 complete [ 3014.509797] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 3022.362818] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3026.968404] vdc: vdc1 vdc9 [ 3032.386477] vde: vde1 vde9 [ 3037.640738] vdf: vdf1 vdf9 [ 3043.170905] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 3043.231756] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 [ 3043.274929] Lustre: lustre-MDT0000: new disk, initializing [ 3043.557956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3043.564931] Lustre: Skipped 2 previous similar messages [ 3043.602675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3045.564670] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3048.960021] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3051.826119] Lustre: lustre-OST0000: new disk, initializing [ 3051.828976] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 3051.831752] Lustre: Skipped 1 previous similar message [ 3053.461610] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3053.466368] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3053.543679] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3054.284136] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3057.598621] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3063.375107] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3063.381079] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3066.516354] Lustre: server umount lustre-OST0000 complete [ 3068.371171] Lustre: server umount lustre-MDT0000 complete [ 3072.960288] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3073.915108] Key type lgssc unregistered [ 3074.027988] LNet: 531:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3074.035933] LNet: Removed LNI 192.168.203.116@tcp [ 3088.279518] alg: No test for adler32 (adler32-zlib) [ 3089.032468] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3089.178673] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 3089.309192] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 3089.313742] LNet: Accept secure, port 988 [ 3090.873319] Key type lgssc registered [ 3091.268487] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3093.802818] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3095.263088] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3096.969531] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3099.137673] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3101.772556] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3104.159889] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3104.757183] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:3 to 0x240000400:33 [ 3106.079634] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3106.569791] Lustre: Setting parameter lustre-OST0000.ost.unknown_param in log lustre-OST0000 [ 3109.759949] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3109.761336] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3109.772195] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3113.468866] Lustre: server umount lustre-OST0000 complete [ 3115.173416] Lustre: server umount lustre-MDT0000 complete [ 3118.643695] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3119.454879] Key type lgssc unregistered [ 3119.538941] LNet: 5190:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3119.544849] LNet: Removed LNI 192.168.203.116@tcp [ 3126.477214] alg: No test for adler32 (adler32-zlib) [ 3127.231392] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3127.318135] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 3127.406575] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 3127.408766] LNet: Accept secure, port 988 [ 3128.957567] Key type lgssc registered [ 3129.280559] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3132.167886] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3149.313034] LustreError: 6347:0:(mgc_request.c:251:do_config_log_add()) MGC192.168.203.116@tcp: failed processing log, type 1: rc = -5 [ 3185.320526] LustreError: 6347:0:(mgc_request.c:251:do_config_log_add()) MGC192.168.203.116@tcp: failed processing log, type 4: rc = -110 [ 3335.370355] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3337.457619] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3339.290803] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3340.304587] Lustre: server umount lustre-OST0000 complete [ 3345.636610] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3346.761134] Key type lgssc unregistered [ 3346.857800] LNet: 8023:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3346.861931] LNet: Removed LNI 192.168.203.116@tcp [ 3363.743264] alg: No test for adler32 (adler32-zlib) [ 3364.504386] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3364.632262] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 3364.711392] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 3364.713644] LNet: Accept secure, port 988 [ 3366.276484] Key type lgssc registered [ 3366.750978] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3389.348198] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3389.534843] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3389.538839] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0xd:0x0]: rc = 0 [ 3389.545013] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x2:0x0]: rc = 0 [ 3389.551989] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3389.558646] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3389.566215] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3389.605573] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xa:0x0]: rc = 0 [ 3389.611322] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3389.617615] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0x9:0x0]: rc = 0 [ 3389.625037] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3389.631168] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3389.637946] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3389.642455] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3389.649602] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3389.664549] Lustre: 14721:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3393.213639] Lustre: server umount MGS complete [ 3393.967638] LustreError: 137-5: lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3394.002654] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3394.072227] Lustre: lustre-MDT0000: Aborting client recovery [ 3394.724537] Lustre: 16190:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3394.732190] Lustre: 16190:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0001: restore index 'nodemap' with [0x200000003:0x7:0x0]: rc = 0 [ 3394.742600] Lustre: 16190:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3394.749587] Lustre: 16190:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x3:0x0]: rc = 0 [ 3394.756951] Lustre: 16190:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3394.763409] Lustre: 16190:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3394.770343] Lustre: 16190:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3394.776674] Lustre: 16190:0:(osd_scrub.c:915:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x4:0x0]: rc = 0 [ 3394.784927] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3394.922749] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3395.006227] Lustre: lustre-MDT0001: Aborting client recovery [ 3395.692955] Lustre: 17007:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3395.697506] Lustre: 17007:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0000: restore index 'nodemap' with [0x200000003:0x2:0x0]: rc = 0 [ 3395.703088] Lustre: 17007:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x4:0x0]: rc = 0 [ 3395.707419] Lustre: 17007:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3395.713171] Lustre: 17007:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x5:0x0]: rc = 0 [ 3395.718425] Lustre: 17007:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3395.723915] Lustre: 17007:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3395.729405] Lustre: 17007:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3395.791188] Lustre: Found index 0 for lustre-OST0000, updating log [ 3395.930631] Lustre: lustre-OST0000: Aborting recovery [ 3396.604348] Lustre: 17915:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3396.609343] Lustre: 17915:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0001: restore index 'nodemap' with [0x200000003:0x2:0x0]: rc = 0 [ 3396.616397] Lustre: 17915:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x5:0x0]: rc = 0 [ 3396.623396] Lustre: 17915:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3396.630101] Lustre: 17915:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3396.635465] Lustre: 17915:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x4:0x0]: rc = 0 [ 3396.640261] Lustre: 17915:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3396.646192] Lustre: 17915:0:(osd_scrub.c:915:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3396.744862] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3396.747412] Lustre: Skipped 1 previous similar message [ 3396.771900] Lustre: lustre-OST0001: Aborting recovery [ 3401.779793] LustreError: 16024:0:(lod_dev.c:525:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [ 3403.747155] Lustre: lustre-OST0000: deleting orphan objects from 0x0:6 to 0x0:33 [ 3403.747974] Lustre: lustre-OST0001: deleting orphan objects from 0x0:6 to 0x0:33 [ 3403.812938] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3403.839812] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3404.788206] Lustre: lustre-OST0001: deleting orphan objects from 0x2c0000400:5 to 0x2c0000400:33 [ 3404.810737] Lustre: lustre-OST0000: deleting orphan objects from 0x280000400:5 to 0x280000400:33 [ 3405.913322] LustreError: 15774:0:(mdt_identity.c:112:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1009us: rc = -2 [ 3419.775111] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3419.775902] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3419.785664] Lustre: Skipped 3 previous similar messages [ 3422.921299] Lustre: server umount lustre-MDT0000 complete [ 3423.720850] LustreError: 17687:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711094725 with bad export cookie 2917359278220276757 [ 3423.722311] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3423.732099] LustreError: 17687:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3424.782752] LustreError: 137-5: lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3424.783364] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3424.800840] LustreError: Skipped 1 previous similar message [ 3424.802035] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3424.802037] Lustre: Skipped 3 previous similar messages [ 3425.925194] Lustre: server umount lustre-MDT0001 complete [ 3432.747981] Lustre: server umount lustre-OST0000 complete [ 3435.679003] Lustre: server umount lustre-OST0001 complete [ 3441.412891] Lustre: DEBUG MARKER: == conf-sanity test 108b: migrate from ZFS to ldiskfs ==== 04:05:42 (1711094742) [ 3441.934886] Lustre: DEBUG MARKER: SKIP: conf-sanity test_108b ldiskfs only test [ 3445.089387] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 04:05:45 (1711094745) [ 3445.855024] Lustre: DEBUG MARKER: SKIP: conf-sanity test_109a LU-8727: no implementation for ZFS [ 3449.249094] Lustre: DEBUG MARKER: == conf-sanity test 109b: test lctl clear_conf one config ========================================================== 04:05:50 (1711094750) [ 3449.929112] Lustre: DEBUG MARKER: SKIP: conf-sanity test_109b LU-8727: no implementation for ZFS [ 3451.575199] Lustre: DEBUG MARKER: SKIP: conf-sanity test_110 skipping ALWAYS excluded test 110 [ 3452.231714] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 3454.427858] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 04:05:55 (1711094755) [ 3456.557997] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3457.838352] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3459.591772] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3461.999387] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3463.886140] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3465.262126] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3466.984149] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:3 to 0x240000400:65 [ 3467.881429] Lustre: lustre-OST0001: new disk, initializing [ 3467.884665] Lustre: srv-lustre-OST0001: No data found on store. Initialize space: rc = -61 [ 3469.851533] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3471.613186] LustreError: 26277:0:(osp_precreate.c:988:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 3472.626971] LustreError: 26277:0:(osp_precreate.c:988:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 3473.631482] LustreError: 26277:0:(osp_precreate.c:988:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 3474.044995] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3474.159170] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3475.195472] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa599800.ost_server_uuid 50 [ 3475.643229] LustreError: 26277:0:(osp_precreate.c:988:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 3475.648231] LustreError: 26277:0:(osp_precreate.c:988:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 3475.844494] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8800aa599800.ost_server_uuid in FULL state after 0 sec [ 3476.903994] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa599800.ost_server_uuid 50 [ 3477.619577] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8800aa599800.ost_server_uuid in FULL state after 0 sec [ 3478.683810] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3478.686704] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3478.735924] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3491.536358] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3491.541879] Lustre: Skipped 1 previous similar message [ 3491.544333] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3491.549139] Lustre: Skipped 1 previous similar message [ 3495.424590] Lustre: server umount lustre-OST0001 complete [ 3496.543593] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3501.555656] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3501.556652] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3501.574938] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3501.578334] Lustre: Skipped 1 previous similar message [ 3504.023334] Lustre: server umount lustre-OST0000 complete [ 3506.308696] Lustre: server umount lustre-MDT0000 complete [ 3511.032802] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3512.320967] Key type lgssc unregistered [ 3512.429275] LNet: 28729:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3512.434667] LNet: Removed LNI 192.168.203.116@tcp [ 3538.370708] alg: No test for adler32 (adler32-zlib) [ 3539.122688] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3539.256555] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 3539.333472] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 3539.337356] LNet: Accept secure, port 988 [ 3540.885135] Key type lgssc registered [ 3541.382212] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3544.539163] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3545.932091] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3547.421646] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3550.805879] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3552.733058] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3555.780890] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3555.793150] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:68 to 0x240000400:97 [ 3555.884444] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3557.881585] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3560.871639] Lustre: lustre-OST0001: deleting orphan objects from 0x280000400:3 to 0x280000400:33 [ 3562.405068] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3565.573612] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 3575.889206] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3575.889690] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3575.898739] Lustre: Skipped 1 previous similar message [ 3578.988871] Lustre: server umount lustre-MDT0000 complete [ 3580.658226] LustreError: 31911:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711094882 with bad export cookie 15483069649955905837 [ 3580.659428] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3580.669838] LustreError: 31911:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3580.690537] Lustre: server umount lustre-OST0000 complete [ 3582.312531] Lustre: server umount lustre-OST0001 complete [ 3586.797900] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3589.501643] Lustre: lustre-MDT0000: set dev_rdonly on this device [ 3589.746334] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3591.035243] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3593.735348] Lustre: lustre-OST0000: set dev_rdonly on this device [ 3593.887217] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3595.712907] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3598.349820] Lustre: lustre-OST0001: set dev_rdonly on this device [ 3600.376922] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3610.350877] Lustre: server umount lustre-MDT0000 complete [ 3612.272169] LustreError: 4874:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711094913 with bad export cookie 15483069649955907055 [ 3612.277778] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3612.296158] Lustre: server umount lustre-OST0000 complete [ 3614.028599] Lustre: server umount lustre-OST0001 complete [ 3619.252404] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 3622.610109] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3622.612277] Lustre: Skipped 1 previous similar message [ 3624.238294] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3628.944109] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3632.620610] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:68 to 0x240000400:129 [ 3632.625140] Lustre: lustre-OST0001: deleting orphan objects from 0x280000400:3 to 0x280000400:65 [ 3633.184834] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3636.290934] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3637.621781] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 3645.763424] Lustre: server umount lustre-MDT0000 complete [ 3647.553509] LustreError: 10627:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711094949 with bad export cookie 15483069649955907657 [ 3647.558811] LustreError: 10627:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3647.565445] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3657.553364] Lustre: 13212:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711094949/real 1711094949] req@ffff88012f14c700 x1794212982975744/t0(0) o39->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711094959 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 3657.593590] Lustre: server umount lustre-OST0000 complete [ 3668.988387] Lustre: 13537:0:(client.c:2338:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1711094960/real 1711094960] req@ffff880093ab9180 x1794212982976192/t0(0) o39->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1711094970 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'umount.0' uid:0 gid:0 [ 3669.087770] Lustre: server umount lustre-OST0001 complete [ 3671.502151] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 3672.179376] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 3674.508736] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 04:09:35 (1711094975) [ 3675.241027] Lustre: DEBUG MARKER: SKIP: conf-sanity test_116 ldiskfs only test [ 3678.957949] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 04:09:39 (1711094979) [ 3681.502274] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3681.505627] Lustre: Skipped 2 previous similar messages [ 3682.724734] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3684.161222] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3688.450616] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3690.150261] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3692.379416] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3692.395247] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:68 to 0x240000400:161 [ 3695.377193] LustreError: 137-5: lustre-OST0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3697.391326] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3702.399769] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3702.400616] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3702.403676] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3702.403678] Lustre: Skipped 1 previous similar message [ 3704.582975] Lustre: server umount lustre-OST0000 complete [ 3706.448674] Lustre: server umount lustre-MDT0000 complete [ 3710.815955] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3711.687798] Key type lgssc unregistered [ 3711.792167] LNet: 18192:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3711.798047] LNet: Removed LNI 192.168.203.116@tcp [ 3735.846607] alg: No test for adler32 (adler32-zlib) [ 3736.600391] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3736.711283] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 3736.799273] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 3736.801713] LNet: Accept secure, port 988 [ 3738.364701] Key type lgssc registered [ 3738.745135] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3741.037900] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3742.425000] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3743.705613] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3745.374464] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3746.207268] Lustre: Failing over lustre-MDT0000 [ 3746.348884] Lustre: server umount lustre-MDT0000 complete [ 3759.513317] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3760.727907] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3763.801303] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mgc.*.mgs_server_uuid [ 3765.957268] Lustre: server umount lustre-MDT0000 complete [ 3768.946272] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3770.123972] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3771.819487] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3772.596674] Lustre: Failing over lustre-MDT0000 [ 3772.728415] Lustre: server umount lustre-MDT0000 complete [ 3785.598020] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3787.058180] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3790.129836] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mgc.*.mgs_server_uuid [ 3792.468755] Lustre: server umount lustre-MDT0000 complete [ 3798.547771] Lustre: DEBUG MARKER: == conf-sanity test 122a: Check OST sequence update ====== 04:11:39 (1711095099) [ 3799.111148] Lustre: DEBUG MARKER: SKIP: conf-sanity test_122a needs >= 2 MDTs [ 3802.328318] Lustre: DEBUG MARKER: == conf-sanity test 123aa: llog_print works with FIDs and simple names ========================================================== 04:11:43 (1711095103) [ 3804.634293] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3805.835028] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3807.538806] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3811.515818] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3813.201232] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3814.666948] LustreError: 137-5: lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3814.684793] Lustre: lustre-OST0000: deleting orphan objects from 0x240000400:68 to 0x240000400:193 [ 3819.402906] LustreError: 137-5: lustre-OST0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3824.413798] LustreError: 137-5: lustre-OST0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3824.421484] LustreError: Skipped 1 previous similar message [ 3826.981993] Lustre: DEBUG MARKER: == conf-sanity test 123ab: llog_print params output values from set_param -P ========================================================== 04:12:07 (1711095127) [ 3827.830400] Lustre: Setting parameter general.jobid_name in log params [ 3828.544523] Lustre: Modifying parameter general.jobid_name in log params [ 3829.423440] LustreError: 137-5: lustre-OST0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3829.432776] LustreError: Skipped 1 previous similar message [ 3833.845033] Lustre: DEBUG MARKER: == conf-sanity test 123ac: llog_print with --start and --end ========================================================== 04:12:14 (1711095134) [ 3834.429593] LustreError: 137-5: lustre-OST0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3834.437815] LustreError: Skipped 1 previous similar message [ 3840.338990] Lustre: DEBUG MARKER: == conf-sanity test 123ad: llog_print shows all records == 04:12:21 (1711095141) [ 3841.100348] Lustre: Setting parameter lustre-OST0000-osc.osc.max_dirty_mb in log lustre-client [ 3844.446256] LustreError: 137-5: lustre-OST0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3844.453070] LustreError: Skipped 3 previous similar messages [ 3846.941525] Lustre: DEBUG MARKER: == conf-sanity test 123ae: llog_cancel can cancel requested record ========================================================== 04:12:27 (1711095147) [ 3847.719319] Lustre: Setting parameter general.osc.*.max_pages_per_rpc in log params [ 3847.721517] Lustre: Skipped 1 previous similar message [ 3855.685758] Lustre: Disabling parameter lustre-OST0000-osc.osc.max_pages_per_rpc in log lustre-client [ 3855.689114] Lustre: Skipped 5 previous similar messages [ 3859.128465] Lustre: DEBUG MARKER: == conf-sanity test 123af: llog_catlist can show all config files correctly ========================================================== 04:12:40 (1711095160) [ 3860.211137] Lustre: *** cfs_fail_loc=131b, val=2*** [ 3861.590116] Lustre: *** cfs_fail_loc=131b, val=2*** [ 3864.479114] LustreError: 137-5: lustre-OST0001: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3864.486522] LustreError: Skipped 7 previous similar messages [ 3867.035363] Lustre: DEBUG MARKER: == conf-sanity test 123ag: llog_print skips values deleted by set_param -P -d ========================================================== 04:12:47 (1711095167) [ 3867.722888] Lustre: Modifying parameter general.jobid_name in log params [ 3867.725369] Lustre: Skipped 2 previous similar messages [ 3874.284681] Lustre: DEBUG MARKER: == conf-sanity test 123ah: del_ost cancels config log entries correctly ========================================================== 04:12:55 (1711095175) [ 3879.775798] LustreError: 11-0: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3879.775811] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3879.776770] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3880.350461] Lustre: server umount lustre-OST0000 complete [ 3881.898325] Lustre: server umount lustre-MDT0000 complete [ 3885.966830] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 3886.774719] Key type lgssc unregistered [ 3886.856519] LNet: 6758:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3886.861960] LNet: Removed LNI 192.168.203.116@tcp [ 3897.109292] alg: No test for adler32 (adler32-zlib) [ 3897.865334] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3897.960802] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 3898.024567] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 3898.028038] LNet: Accept secure, port 988 [ 3899.566393] Key type lgssc registered [ 3899.867954] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3902.613841] vdc: vdc1 vdc9 [ 3906.079899] vde: vde1 vde9 [ 3909.712821] vdf: vdf1 vdf9 [ 3912.999786] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3914.221624] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 3914.272309] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 [ 3914.309919] Lustre: lustre-MDT0000: new disk, initializing [ 3914.486309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3914.516265] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3915.868842] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3918.722371] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3921.331495] Lustre: lustre-OST0000: new disk, initializing [ 3921.334982] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 3921.337758] Lustre: Skipped 1 previous similar message [ 3921.386473] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3922.080416] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3922.084183] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3922.148233] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3923.240312] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3926.158589] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3927.368936] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3927.447613] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3932.065915] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3932.077420] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3934.321661] Lustre: server umount lustre-OST0000 complete [ 3936.031200] Lustre: server umount lustre-MDT0000 complete [ 3941.936033] Lustre: DEBUG MARKER: == conf-sanity test 123ai: llog_print display all non skipped records ========================================================== 04:14:02 (1711095242) [ 3944.261279] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3945.624315] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3947.271892] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3949.409786] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3951.724850] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 3953.904305] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3961.980368] LustreError: 17663:0:(llog.c:436:llog_init_handle()) llog type is not specified! [ 3962.324756] Lustre: Modifying parameter general.timeout in log params [ 3963.574663] Lustre: Modifying parameter general.timeout in log params [ 3963.576738] Lustre: Skipped 3 previous similar messages [ 3965.881018] Lustre: Modifying parameter general.timeout in log params [ 3965.883047] Lustre: Skipped 6 previous similar messages [ 3970.165018] Lustre: Modifying parameter general.timeout in log params [ 3970.167300] Lustre: Skipped 11 previous similar messages [ 3978.377786] Lustre: Modifying parameter general.timeout in log params [ 3978.380029] Lustre: Skipped 21 previous similar messages [ 3994.578474] Lustre: Modifying parameter general.timeout in log params [ 3994.580914] Lustre: Skipped 48 previous similar messages [ 4012.277445] Lustre: DEBUG MARKER: == conf-sanity test 123F: clear and reset all parameters using set_param -F ========================================================== 04:15:13 (1711095313) [ 4020.480989] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4020.489940] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4021.888303] Lustre: server umount lustre-MDT0000 complete [ 4023.427952] LustreError: 15906:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711095325 with bad export cookie 13955950697227725823 [ 4023.430840] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4023.440869] LustreError: 15906:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4023.515132] Lustre: server umount lustre-OST0000 complete [ 4033.050784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4034.180235] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4035.722735] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4039.690923] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4041.313223] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4046.972205] Lustre: Setting parameter lustre.mdt.lustre-MDT0000.identity_upcall in log params [ 4046.975183] Lustre: Skipped 36 previous similar messages [ 4052.766998] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4052.774647] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4054.787317] Lustre: server umount lustre-OST0000 complete [ 4056.465824] Lustre: server umount lustre-MDT0000 complete [ 4060.392774] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4061.081641] Key type lgssc unregistered [ 4061.179794] LNet: 411:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4061.183372] LNet: Removed LNI 192.168.203.116@tcp [ 4071.921914] alg: No test for adler32 (adler32-zlib) [ 4072.675398] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4072.764532] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 4072.825347] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4072.827294] LNet: Accept secure, port 988 [ 4074.366510] Key type lgssc registered [ 4074.708515] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4076.600952] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4077.969846] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4079.182092] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4080.833213] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4083.331432] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4085.355322] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4086.963538] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4095.908666] Lustre: Modifying parameter general.jobid_var in log params [ 4103.344932] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4103.353889] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4104.586510] Lustre: server umount lustre-MDT0000 complete [ 4106.365186] LustreError: 2251:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1711095407 with bad export cookie 11893735992477143155 [ 4106.368542] LustreError: 166-1: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4106.369060] LustreError: 2263:0:(mgc_request.c:621:do_requeue()) failed processing log: -5 [ 4106.377022] LustreError: 2251:0:(ldlm_lockd.c:2594:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4106.437564] Lustre: server umount lustre-OST0000 complete [ 4114.429306] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4115.570463] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4116.957378] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4118.969884] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4120.754634] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4122.299142] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4127.890127] Lustre: Modifying parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 4133.966922] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4133.972573] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4136.200688] Lustre: server umount lustre-OST0000 complete [ 4138.094808] Lustre: server umount lustre-MDT0000 complete [ 4142.067433] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing unload_modules_local [ 4142.756651] Key type lgssc unregistered [ 4142.835719] LNet: 9890:0:(lib-ptl.c:990:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4142.839817] LNet: Removed LNI 192.168.203.116@tcp [ 4160.631594] alg: No test for adler32 (adler32-zlib) [ 4161.383356] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4161.470488] Lustre: Lustre: Build Version: 2.15.61_139_g36c911c [ 4161.548082] LNet: Added LNI 192.168.203.116@tcp [8/256/0/180] [ 4161.550817] LNet: Accept secure, port 988 [ 4163.093935] Key type lgssc registered [ 4163.361022] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4163.570392] LustreError: 13254:0:(lu_object.c:1416:lu_context_key_register()) cfs_race id 60d sleeping [ 4169.568439] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4172.372699] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4184.369931] LustreError: 13254:0:(lu_object.c:1416:lu_context_key_register()) cfs_fail_race id 60d awake: rc=0 [ 4185.704919] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4186.714708] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4191.805800] Lustre: DEBUG MARKER: == conf-sanity test 127: direct io overwrite on full ost ========================================================== 04:18:12 (1711095492) [ 4192.370510] Lustre: DEBUG MARKER: SKIP: conf-sanity test_127 ldiskfs only test [ 4195.323154] Lustre: DEBUG MARKER: == conf-sanity test 128: Force using remote logs with --nolocallogs ========================================================== 04:18:16 (1711095496) [ 4195.903040] Lustre: DEBUG MARKER: SKIP: conf-sanity test_128 need separate mgs device [ 4198.816200] Lustre: DEBUG MARKER: == conf-sanity test 129: attempt to connect an OST with the same index should fail ========================================================== 04:18:19 (1711095499) [ 4199.758989] Lustre: server umount lustre-MDT0000 complete [ 4205.189087] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4206.433842] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4207.907446] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4210.073153] vde: vde1 vde9 [ 4214.140787] LustreError: 140-5: Server lustre-OST0000 requested index 0, but that index is already in use. Use --writeconf to force [ 4214.144813] LustreError: 17629:0:(mgs_handler.c:515:mgs_target_reg()) Failed to write lustre-OST0000 log (-98) [ 4214.150911] LustreError: 160-7: lustre-OST0000: the MGS refuses to allow this server to start: rc = -98. Please see messages on the MGS. [ 4214.155624] LustreError: 19415:0:(tgt_mount.c:2223:server_fill_super()) Unable to start targets: -98 [ 4214.158878] LustreError: 19415:0:(tgt_mount.c:1755:server_put_super()) no obd lustre-OST0000 [ 4214.161661] LustreError: 19415:0:(tgt_mount.c:135:server_deregister_mount()) lustre-OST0000 not registered [ 4214.262337] Lustre: server umount lustre-OST0000 complete [ 4214.264684] LustreError: 19415:0:(super25.c:188:lustre_fill_super()) llite: Unable to mount : rc = -98 [ 4216.559418] LustreError: 140-5: Server lustre-OST0000 requested index 0, but that index is already in use. Use --writeconf to force [ 4216.564838] LustreError: 17630:0:(mgs_handler.c:515:mgs_target_reg()) Failed to write lustre-OST0000 log (-98) [ 4216.573687] LustreError: 160-7: lustre-OST0000: the MGS refuses to allow this server to start: rc = -98. Please see messages on the MGS. [ 4216.580026] LustreError: 20230:0:(tgt_mount.c:2223:server_fill_super()) Unable to start targets: -98 [ 4216.585258] LustreError: 20230:0:(tgt_mount.c:1755:server_put_super()) no obd lustre-OST0000 [ 4216.588804] LustreError: 20230:0:(tgt_mount.c:135:server_deregister_mount()) lustre-OST0000 not registered [ 4216.687328] Lustre: server umount lustre-OST0000 complete [ 4216.689982] LustreError: 20230:0:(super25.c:188:lustre_fill_super()) llite: Unable to mount : rc = -98 [ 4219.695745] Lustre: MGS: Regenerating lustre-OST0000 log by user request: rc = 0 [ 4219.698142] Lustre: Found index 0 for lustre-OST0000, updating log [ 4219.702749] Lustre: Client log for lustre-OST0000 was not updated; writeconf the MDT first to regenerate it. [ 4219.782650] Lustre: lustre-OST0000: new disk, initializing [ 4219.785750] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 4219.901506] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4221.497489] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4222.832418] Lustre: Failing over lustre-OST0000 [ 4222.919412] Lustre: server umount lustre-OST0000 complete [ 4224.447040] Lustre: server umount lustre-MDT0000 complete [ 4229.660466] Lustre: DEBUG MARKER: == conf-sanity test 130: re-register an MDT after writeconf ========================================================== 04:18:50 (1711095530) [ 4230.266426] Lustre: DEBUG MARKER: SKIP: conf-sanity test_130 needs >= 2 MDTs [ 4233.077272] Lustre: DEBUG MARKER: == conf-sanity test 131: MDT backup restore with project ID ========================================================== 04:18:54 (1711095534) [ 4233.556021] Lustre: DEBUG MARKER: SKIP: conf-sanity test_131 ldiskfs only test [ 4236.048405] Lustre: DEBUG MARKER: == conf-sanity test 132: hsm_actions processed after failover ========================================================== 04:18:57 (1711095537) [ 4239.996905] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 4242.021040] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4244.939831] vdc: vdc1 vdc9 [ 4248.135067] vde: vde1 vde9 [ 4251.858348] vdf: vdf1 vdf9 [ 4255.427960] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 4255.456951] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 [ 4255.492400] Lustre: lustre-MDT0000: new disk, initializing [ 4255.645154] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4255.677430] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4256.978889] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4259.943818] Lustre: server umount lustre-MDT0000 complete [ 4263.368484] Lustre: Found index 0 for lustre-MDT0000, updating log [ 4263.379604] Lustre: Setting parameter lustre-MDT0000.mdt.hsm_control in log lustre-MDT0000 [ 4263.490261] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4264.532286] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4266.367695] Lustre: server umount lustre-MDT0000 complete [ 4271.231092] Lustre: DEBUG MARKER: == conf-sanity test 133: stripe QOS: free space balance in a pool ========================================================== 04:19:32 (1711095572) [ 4271.719030] Lustre: DEBUG MARKER: SKIP: conf-sanity test_133 needs >= 4 OSTs [ 4274.374590] Lustre: DEBUG MARKER: == conf-sanity test 134: check_iam works without faults == 04:19:35 (1711095575) [ 4274.914532] Lustre: DEBUG MARKER: SKIP: conf-sanity test_134 ldiskfs only test [ 4277.804771] Lustre: DEBUG MARKER: == conf-sanity test 135: check the behavior when changelog is wrapped around ========================================================== 04:19:38 (1711095578) [ 4282.471476] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_hostid [ 4284.830055] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local [ 4287.808517] vdc: vdc1 vdc9 [ 4291.174296] vde: vde1 vde9 [ 4294.967558] vdf: vdf1 vdf9 [ 4298.537259] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 4298.539759] Lustre: Skipped 1 previous similar message [ 4298.576007] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space: rc = -61 [ 4298.578904] Lustre: Skipped 1 previous similar message [ 4298.614155] Lustre: lustre-MDT0000: new disk, initializing [ 4298.741022] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4298.769900] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4299.804179] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4302.363918] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4304.289027] Lustre: lustre-OST0000: new disk, initializing [ 4304.291196] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 4304.293750] Lustre: Skipped 1 previous similar message [ 4305.866096] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug -1 all [ 4307.674961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4307.681742] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4307.740027] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4308.507087] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4312.090008] Lustre: lustre-MDD0000: changelog on [ 4379.110495] Lustre: lustre-MDT0000: Client ce74609a-7628-42b9-b29e-c6dcebb59e91 (at 192.168.203.16@tcp) reconnecting [ 4389.596796] Lustre: DEBUG MARKER: conf-sanity test_135: @@@@@@ FAIL: creation thread is running. Is changelog reader stuck?