[ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup subsys cpuacct [ 0.000000] Linux version 3.10.0-7.9-debug (green@centos7-base) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) ) #1 SMP Sat Mar 26 23:28:42 EDT 2022 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:centos7:ext4:ro:-p,-b4096 ro crashkernel=128M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000013edfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000000] e820: last_pfn = 0x13ee00 max_arch_pfn = 0x400000000 [ 0.000000] MTRR default type: write-back [ 0.000000] MTRR fixed ranges enabled: [ 0.000000] 00000-9FFFF write-back [ 0.000000] A0000-BFFFF uncachable [ 0.000000] C0000-FFFFF write-protect [ 0.000000] MTRR variable ranges enabled: [ 0.000000] 0 base 0000C0000000 mask 3FFFC0000000 uncachable [ 0.000000] 1 disabled [ 0.000000] 2 disabled [ 0.000000] 3 disabled [ 0.000000] 4 disabled [ 0.000000] 5 disabled [ 0.000000] 6 disabled [ 0.000000] 7 disabled [ 0.000000] PAT configuration [0-7]: WB WC UC- UC WB WP UC- UC [ 0.000000] e820: last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] mapped at [ffffffffff200b30] [ 0.000000] Base memory trampoline at [ffff880000099000] 99000 size 24576 [ 0.000000] Using GB pages for direct mapping [ 0.000000] BRK [0x023c1000, 0x023c1fff] PGTABLE [ 0.000000] BRK [0x023c2000, 0x023c2fff] PGTABLE [ 0.000000] BRK [0x023c3000, 0x023c3fff] PGTABLE [ 0.000000] BRK [0x023c4000, 0x023c4fff] PGTABLE [ 0.000000] BRK [0x023c5000, 0x023c5fff] PGTABLE [ 0.000000] BRK [0x023c6000, 0x023c6fff] PGTABLE [ 0.000000] RAMDISK: [mem 0xbc2e2000-0xbffbffff] [ 0.000000] Early table checksum verification disabled [ 0.000000] ACPI: RSDP 00000000000f5950 00014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 00000000bffe1bb7 00034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 00000000bffe1a53 00074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 00000000bffe0040 01A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 00000000bffe0000 00040 [ 0.000000] ACPI: APIC 00000000bffe1ac7 00090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 00000000bffe1b57 00038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 00000000bffe1b8f 00028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000013edfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x13e5e3000-0x13e609fff] [ 0.000000] Reserving 128MB of memory at 768MB for crashkernel (System RAM: 4077MB) [ 0.000000] kvm-clock: cpu 0, msr 1:3e592001, primary cpu clock [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 371213562 cycles [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x00001000-0x00ffffff] [ 0.000000] DMA32 [mem 0x01000000-0xffffffff] [ 0.000000] Normal [mem 0x100000000-0x13edfffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x00001000-0x0009efff] [ 0.000000] node 0: [mem 0x00100000-0xbffcdfff] [ 0.000000] node 0: [mem 0x100000000-0x13edfffff] [ 0.000000] Initmem setup node 0 [mem 0x00001000-0x13edfffff] [ 0.000000] On node 0 totalpages: 1043820 [ 0.000000] DMA zone: 64 pages used for memmap [ 0.000000] DMA zone: 21 pages reserved [ 0.000000] DMA zone: 3998 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 12224 pages used for memmap [ 0.000000] DMA32 zone: 782286 pages, LIFO batch:31 [ 0.000000] Normal zone: 4024 pages used for memmap [ 0.000000] Normal zone: 257536 pages, LIFO batch:31 [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] ACPI: IRQ0 used by override. [ 0.000000] ACPI: IRQ5 used by override. [ 0.000000] ACPI: IRQ9 used by override. [ 0.000000] ACPI: IRQ10 used by override. [ 0.000000] ACPI: IRQ11 used by override. [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] e820: [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 38 pages/cpu s115176 r8192 d32280 u524288 [ 0.000000] pcpu-alloc: s115176 r8192 d32280 u524288 alloc=1*2097152 [ 0.000000] pcpu-alloc: [0] 0 1 2 3 [ 0.000000] KVM setup async PF for cpu 0 [ 0.000000] kvm-stealtime: cpu 0, msr 13e2135c0 [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes) [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 1027487 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:centos7:ext4:ro:-p,-b4096 ro crashkernel=128M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] audit: disabled (until reboot) [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] x86/fpu: xstate_offset[2]: 0240, xstate_sizes[2]: 0100 [ 0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form [ 0.000000] Memory: 3820268k/5224448k available (8172k kernel code, 1049168k absent, 355012k reserved, 5773k data, 2532k init) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=4. [ 0.000000] Offload RCU callbacks from all CPUs [ 0.000000] Offload RCU callbacks from CPUs: 0-3. [ 0.000000] NR_IRQS:327936 nr_irqs:456 0 [ 0.000000] Console: colour *CGA 80x25 [ 0.000000] console [ttyS1] enabled [ 0.000000] allocated 25165824 bytes of page_cgroup [ 0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ODEBUG: 73 of 73 active objects replaced [ 0.000000] hpet clockevent registered [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.444258] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.447303] pid_max: default: 32768 minimum: 301 [ 0.448925] Security Framework initialized [ 0.450743] SELinux: Initializing. [ 0.451733] SELinux: Starting in permissive mode [ 0.453321] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) [ 0.458247] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) [ 0.461316] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes) [ 0.463818] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes) [ 0.467143] Initializing cgroup subsys memory [ 0.470043] Initializing cgroup subsys devices [ 0.471911] Initializing cgroup subsys freezer [ 0.473930] Initializing cgroup subsys net_cls [ 0.475099] Initializing cgroup subsys blkio [ 0.475962] Initializing cgroup subsys perf_event [ 0.477386] Initializing cgroup subsys hugetlb [ 0.478823] Initializing cgroup subsys pids [ 0.480049] Initializing cgroup subsys net_prio [ 0.481880] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.485901] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.487734] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.489721] tlb_flushall_shift: 6 [ 0.490845] FEATURE SPEC_CTRL Present [ 0.492283] FEATURE IBPB_SUPPORT Present [ 0.494519] Spectre V2 : Enabling Indirect Branch Prediction Barrier [ 0.496471] Spectre V2 : Vulnerable [ 0.498738] Speculative Store Bypass: Vulnerable [ 0.501170] debug: unmapping init [mem 0xffffffff82019000-0xffffffff8201ffff] [ 0.510817] ACPI: Core revision 20130517 [ 0.513788] ACPI: All ACPI Tables successfully acquired [ 0.515523] ftrace: allocating 30294 entries in 119 pages [ 0.586985] Enabling x2apic [ 0.587999] Enabled x2apic [ 0.589640] Switched APIC routing to physical x2apic. [ 0.593868] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.596055] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (fam: 06, model: 3e, stepping: 04) [ 0.600403] TSC deadline timer enabled [ 0.600571] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.602981] ... version: 2 [ 0.605880] ... bit width: 48 [ 0.608113] ... generic registers: 4 [ 0.609321] ... value mask: 0000ffffffffffff [ 0.611092] ... max period: 00007fffffffffff [ 0.612826] ... fixed-purpose events: 3 [ 0.614389] ... event mask: 000000070000000f [ 0.618546] KVM setup paravirtual spinlock [ 0.624096] kvm-clock: cpu 1, msr 1:3e592041, secondary cpu clock [ 0.637473] KVM setup async PF for cpu 1 [ 0.638851] kvm-stealtime: cpu 1, msr 13e2935c0 [ 0.643395] kvm-clock: cpu 2, msr 1:3e592081, secondary cpu clock [ 0.646768] KVM setup async PF for cpu 2 [ 0.621827] smpboot: Booting Node 0, Processors #1 #2 #3 OK [ 0.647881] kvm-clock: cpu 3, msr 1:3e5920c1, secondary cpu clock [ 0.651728] kvm-stealtime: cpu 2, msr 13e3135c0 [ 0.653770] Brought up 4 CPUs [ 0.653780] KVM setup async PF for cpu 3 [ 0.653787] kvm-stealtime: cpu 3, msr 13e3935c0 [ 0.657400] smpboot: Max logical packages: 1 [ 0.658808] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.663694] devtmpfs: initialized [ 0.665027] x86/mm: Memory block size: 128MB [ 0.669973] EVM: security.selinux [ 0.671052] EVM: security.ima [ 0.672006] EVM: security.capability [ 0.678697] atomic64 test passed for x86-64 platform with CX8 and with SSE [ 0.681212] NET: Registered protocol family 16 [ 0.682948] cpuidle: using governor haltpoll [ 0.684985] ACPI: bus type PCI registered [ 0.686383] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.688760] PCI: Using configuration type 1 for base access [ 0.690639] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.702880] ACPI: Added _OSI(Module Device) [ 0.704051] ACPI: Added _OSI(Processor Device) [ 0.705934] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.707829] ACPI: Added _OSI(Processor Aggregator Device) [ 0.709772] ACPI: Added _OSI(Linux-Dell-Video) [ 0.712412] ACPI: EC: Look up EC in DSDT [ 0.715260] ACPI: Interpreter enabled [ 0.716819] ACPI: (supports S0 S3 S4 S5) [ 0.718000] ACPI: Using IOAPIC for interrupt routing [ 0.719936] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.723543] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.731658] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.734142] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI] [ 0.736755] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM [ 0.739402] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.744113] acpiphp: Slot [2] registered [ 0.745624] acpiphp: Slot [3] registered [ 0.747075] acpiphp: Slot [4] registered [ 0.748665] acpiphp: Slot [5] registered [ 0.750307] acpiphp: Slot [6] registered [ 0.751844] acpiphp: Slot [7] registered [ 0.753477] acpiphp: Slot [8] registered [ 0.755200] acpiphp: Slot [9] registered [ 0.756387] acpiphp: Slot [10] registered [ 0.758112] acpiphp: Slot [11] registered [ 0.759602] acpiphp: Slot [12] registered [ 0.760921] acpiphp: Slot [13] registered [ 0.762514] acpiphp: Slot [14] registered [ 0.764307] acpiphp: Slot [15] registered [ 0.766891] acpiphp: Slot [16] registered [ 0.770366] acpiphp: Slot [17] registered [ 0.771878] acpiphp: Slot [18] registered [ 0.773974] acpiphp: Slot [19] registered [ 0.777003] acpiphp: Slot [20] registered [ 0.780162] acpiphp: Slot [21] registered [ 0.782402] acpiphp: Slot [22] registered [ 0.784271] acpiphp: Slot [23] registered [ 0.785630] acpiphp: Slot [24] registered [ 0.788189] acpiphp: Slot [25] registered [ 0.790903] acpiphp: Slot [26] registered [ 0.792570] acpiphp: Slot [27] registered [ 0.793899] acpiphp: Slot [28] registered [ 0.795388] acpiphp: Slot [29] registered [ 0.796633] acpiphp: Slot [30] registered [ 0.798684] acpiphp: Slot [31] registered [ 0.800194] PCI host bridge to bus 0000:00 [ 0.801706] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.804021] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.806515] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.808970] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.811275] pci_bus 0000:00: root bus resource [mem 0x140000000-0x1bfffffff window] [ 0.812749] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.814061] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.814898] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.815897] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.825616] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.829152] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.831895] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.835206] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.841888] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.844527] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.844954] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.847127] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.849615] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.852470] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.863533] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.866813] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.868032] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.878721] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.888690] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.913331] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.919212] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.930812] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.937944] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.970587] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.984021] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.990559] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.993364] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 1.004558] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 1.008123] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 1.013197] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 1.017803] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 1.026260] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 1.033814] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 1.037918] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 1.043317] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 1.072658] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 1.082059] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 1.087298] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 1.090429] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.116598] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 1.123662] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11) [ 1.126383] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11) [ 1.128296] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11) [ 1.130576] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11) [ 1.133027] ACPI: PCI Interrupt Link [LNKS] (IRQs *9) [ 1.141463] vgaarb: loaded [ 1.144180] SCSI subsystem initialized [ 1.146142] ACPI: bus type USB registered [ 1.147459] usbcore: registered new interface driver usbfs [ 1.149205] usbcore: registered new interface driver hub [ 1.151239] usbcore: registered new device driver usb [ 1.153800] PCI: Using ACPI for IRQ routing [ 1.156810] PCI: pci_cache_line_size set to 64 bytes [ 1.157394] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] [ 1.157405] e820: reserve RAM buffer [mem 0xbffce000-0xbfffffff] [ 1.157406] e820: reserve RAM buffer [mem 0x13ee00000-0x13fffffff] [ 1.158111] NetLabel: Initializing [ 1.159087] NetLabel: domain hash size = 128 [ 1.160754] NetLabel: protocols = UNLABELED CIPSOv4 [ 1.163286] NetLabel: unlabeled traffic allowed by default [ 1.166651] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.168465] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.173821] amd_nb: Cannot enumerate AMD northbridges [ 1.177530] Switched to clocksource kvm-clock [ 1.198929] pnp: PnP ACPI init [ 1.200182] ACPI: bus type PNP registered [ 1.201893] pnp 00:00: Plug and Play ACPI device, IDs PNP0303 (active) [ 1.201983] pnp 00:01: Plug and Play ACPI device, IDs PNP0f13 (active) [ 1.202029] pnp 00:02: [dma 2] [ 1.202070] pnp 00:02: Plug and Play ACPI device, IDs PNP0700 (active) [ 1.202145] pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active) [ 1.202226] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active) [ 1.202299] pnp 00:05: Plug and Play ACPI device, IDs PNP0b00 (active) [ 1.202715] pnp: PnP ACPI: found 6 devices [ 1.204262] ACPI: bus type PNP unregistered [ 1.220841] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.220850] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.220853] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.220855] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.220858] pci_bus 0000:00: resource 8 [mem 0x140000000-0x1bfffffff window] [ 1.220955] NET: Registered protocol family 2 [ 1.222700] TCP established hash table entries: 32768 (order: 6, 262144 bytes) [ 1.225850] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes) [ 1.230588] TCP: Hash tables configured (established 32768 bind 32768) [ 1.235101] TCP: reno registered [ 1.237405] UDP hash table entries: 2048 (order: 5, 196608 bytes) [ 1.241344] UDP-Lite hash table entries: 2048 (order: 5, 196608 bytes) [ 1.244111] NET: Registered protocol family 1 [ 1.246782] RPC: Registered named UNIX socket transport module. [ 1.249152] RPC: Registered udp transport module. [ 1.250839] RPC: Registered tcp transport module. [ 1.252615] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.255280] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.257185] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.259014] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.261007] PCI: CLS 0 bytes, default 64 [ 1.261714] Unpacking initramfs... [ 2.658168] debug: unmapping init [mem 0xffff8800bc2e2000-0xffff8800bffbffff] [ 2.661834] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.663947] software IO TLB [mem 0xb82e2000-0xbc2e2000] (64MB) mapped at [ffff8800b82e2000-ffff8800bc2e1fff] [ 2.667069] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 10737418240 ms ovfl timer [ 2.669136] RAPL PMU: hw unit of domain pp0-core 2^-0 Joules [ 2.670967] RAPL PMU: hw unit of domain package 2^-0 Joules [ 2.672940] RAPL PMU: hw unit of domain dram 2^-0 Joules [ 2.676051] cryptomgr_test (50) used greatest stack depth: 14480 bytes left [ 2.676721] futex hash table entries: 1024 (order: 4, 65536 bytes) [ 2.676805] Initialise system trusted keyring [ 2.708332] HugeTLB registered 1 GB page size, pre-allocated 0 pages [ 2.711558] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 2.718712] zpool: loaded [ 2.719765] zbud: loaded [ 2.720923] VFS: Disk quotas dquot_6.6.0 [ 2.723244] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 2.726764] NFS: Registering the id_resolver key type [ 2.728599] Key type id_resolver registered [ 2.729662] Key type id_legacy registered [ 2.730960] nfs4filelayout_init: NFSv4 File Layout Driver Registering... [ 2.733326] Key type big_key registered [ 2.734371] SELinux: Registering netfilter hooks [ 2.735884] cryptomgr_test (57) used greatest stack depth: 14048 bytes left [ 2.738574] cryptomgr_test (58) used greatest stack depth: 13968 bytes left [ 2.741289] NET: Registered protocol family 38 [ 2.742867] Key type asymmetric registered [ 2.743933] Asymmetric key parser 'x509' registered [ 2.745528] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250) [ 2.748115] io scheduler noop registered [ 2.749300] io scheduler deadline registered (default) [ 2.750965] io scheduler cfq registered [ 2.752734] io scheduler mq-deadline registered [ 2.754061] io scheduler kyber registered [ 2.757699] pci_hotplug: PCI Hot Plug PCI Core version: 0.5 [ 2.759594] pciehp: PCI Express Hot Plug Controller Driver version: 0.4 [ 2.761671] intel_idle: does not run on family 6 model 62 [ 2.761801] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.764110] ACPI: Power Button [PWRF] [ 2.765624] GHES: HEST is not enabled! [ 2.821336] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10 [ 2.881415] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 11 [ 3.009171] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11 [ 3.070358] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 10 [ 3.191053] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.220873] 00:03: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A [ 3.250943] 00:04: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A [ 3.254993] Non-volatile memory driver v1.3 [ 3.256635] Linux agpgart interface v0.103 [ 3.258321] crash memory driver: version 1.1 [ 3.260087] nbd: registered device at major 43 [ 3.265453] virtio-pci 0000:00:05.0: irq 24 for MSI/MSI-X [ 3.265512] virtio-pci 0000:00:05.0: irq 25 for MSI/MSI-X [ 3.265557] virtio-pci 0000:00:05.0: irq 26 for MSI/MSI-X [ 3.265589] virtio-pci 0000:00:05.0: irq 27 for MSI/MSI-X [ 3.265637] virtio-pci 0000:00:05.0: irq 28 for MSI/MSI-X [ 3.271578] virtio_blk virtio1: [vda] 115512 512-byte logical blocks (59.1 MB/56.4 MiB) [ 3.277364] virtio-pci 0000:00:06.0: irq 29 for MSI/MSI-X [ 3.277421] virtio-pci 0000:00:06.0: irq 30 for MSI/MSI-X [ 3.277466] virtio-pci 0000:00:06.0: irq 31 for MSI/MSI-X [ 3.277530] virtio-pci 0000:00:06.0: irq 32 for MSI/MSI-X [ 3.277574] virtio-pci 0000:00:06.0: irq 33 for MSI/MSI-X [ 3.284584] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.291799] virtio-pci 0000:00:07.0: irq 34 for MSI/MSI-X [ 3.291849] virtio-pci 0000:00:07.0: irq 35 for MSI/MSI-X [ 3.291887] virtio-pci 0000:00:07.0: irq 36 for MSI/MSI-X [ 3.291911] virtio-pci 0000:00:07.0: irq 37 for MSI/MSI-X [ 3.291933] virtio-pci 0000:00:07.0: irq 38 for MSI/MSI-X [ 3.297128] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.303555] virtio-pci 0000:00:08.0: irq 39 for MSI/MSI-X [ 3.303648] virtio-pci 0000:00:08.0: irq 40 for MSI/MSI-X [ 3.303678] virtio-pci 0000:00:08.0: irq 41 for MSI/MSI-X [ 3.303703] virtio-pci 0000:00:08.0: irq 42 for MSI/MSI-X [ 3.303733] virtio-pci 0000:00:08.0: irq 43 for MSI/MSI-X [ 3.310810] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.317855] virtio-pci 0000:00:09.0: irq 44 for MSI/MSI-X [ 3.317904] virtio-pci 0000:00:09.0: irq 45 for MSI/MSI-X [ 3.317945] virtio-pci 0000:00:09.0: irq 46 for MSI/MSI-X [ 3.317985] virtio-pci 0000:00:09.0: irq 47 for MSI/MSI-X [ 3.318028] virtio-pci 0000:00:09.0: irq 48 for MSI/MSI-X [ 3.323903] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.331983] virtio-pci 0000:00:0a.0: irq 49 for MSI/MSI-X [ 3.332028] virtio-pci 0000:00:0a.0: irq 50 for MSI/MSI-X [ 3.332063] virtio-pci 0000:00:0a.0: irq 51 for MSI/MSI-X [ 3.332105] virtio-pci 0000:00:0a.0: irq 52 for MSI/MSI-X [ 3.332147] virtio-pci 0000:00:0a.0: irq 53 for MSI/MSI-X [ 3.338622] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.346204] rdac: device handler registered [ 3.347820] hp_sw: device handler registered [ 3.349171] emc: device handler registered [ 3.350540] libphy: Fixed MDIO Bus: probed [ 3.356763] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 3.358832] ehci-pci: EHCI PCI platform driver [ 3.360223] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 3.362816] ohci-pci: OHCI PCI platform driver [ 3.364162] uhci_hcd: USB Universal Host Controller Interface driver [ 3.366164] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.369914] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.371519] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.373737] mousedev: PS/2 mouse device common for all mice [ 3.375980] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.379536] rtc_cmos 00:05: RTC can wake from S4 [ 3.383789] rtc_cmos 00:05: rtc core: registered rtc_cmos as rtc0 [ 3.386084] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.389721] hidraw: raw HID events driver (C) Jiri Kosina [ 3.391716] usbcore: registered new interface driver usbhid [ 3.393393] usbhid: USB HID core driver [ 3.394824] drop_monitor: Initializing network drop monitor service [ 3.397146] Netfilter messages via NETLINK v0.30. [ 3.398788] TCP: cubic registered [ 3.399963] Initializing XFRM netlink socket [ 3.401710] NET: Registered protocol family 10 [ 3.404057] NET: Registered protocol family 17 [ 3.405555] Key type dns_resolver registered [ 3.407343] mce: Using 10 MCE banks [ 3.408752] Loading compiled-in X.509 certificates [ 3.411241] Loaded X.509 cert 'Magrathea: Glacier signing key: e34d0e1b7fcf5b414cce75d36d8482945c781ed6' [ 3.414097] registered taskstats version 1 [ 3.418506] modprobe (71) used greatest stack depth: 13456 bytes left [ 3.424052] Key type trusted registered [ 3.430965] Key type encrypted registered [ 3.432376] IMA: No TPM chip found, activating TPM-bypass! (rc=-19) [ 3.436263] BERT: Boot Error Record Table support is disabled. Enable it by using bert_enable as kernel parameter. [ 3.439966] rtc_cmos 00:05: setting system clock to 2025-04-08 19:26:50 UTC (1744140410) [ 3.443190] debug: unmapping init [mem 0xffffffff81da0000-0xffffffff82018fff] [ 3.445815] Write protecting the kernel read-only data: 12288k [ 3.447875] debug: unmapping init [mem 0xffff8800017fe000-0xffff8800017fffff] [ 3.450065] debug: unmapping init [mem 0xffff880001b9b000-0xffff880001bfffff] [ 3.458639] random: systemd: uninitialized urandom read (16 bytes read) [ 3.462016] random: systemd: uninitialized urandom read (16 bytes read) [ 3.464582] random: systemd: uninitialized urandom read (16 bytes read) [ 3.469108] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN) [ 3.475721] systemd[1]: Detected virtualization kvm. [ 3.477757] systemd[1]: Detected architecture x86-64. [ 3.479752] systemd[1]: Running in initial RAM disk. [ 3.485662] systemd[1]: No hostname configured. [ 3.487177] systemd[1]: Set hostname to . [ 3.489310] random: systemd: uninitialized urandom read (16 bytes read) [ 3.491762] systemd[1]: Initializing machine ID from random generator. [ 3.510162] mkdir (89) used greatest stack depth: 13424 bytes left [ 3.540770] dracut-rootfs-g (85) used greatest stack depth: 13264 bytes left [ 3.543669] random: systemd: uninitialized urandom read (16 bytes read) [ 3.545772] random: systemd: uninitialized urandom read (16 bytes read) [ 3.547973] random: systemd: uninitialized urandom read (16 bytes read) [ 3.550173] random: systemd: uninitialized urandom read (16 bytes read) [ 3.553297] random: systemd: uninitialized urandom read (16 bytes read) [ 3.555645] random: systemd: uninitialized urandom read (16 bytes read) [ 3.565682] systemd[1]: Reached target Timers. [ 3.569266] systemd[1]: Created slice Root Slice. [ 3.572875] systemd[1]: Listening on udev Kernel Socket. [ 3.576577] systemd[1]: Created slice System Slice. [ 3.579254] systemd[1]: Reached target Swap. [ 3.582324] systemd[1]: Listening on Journal Socket. [ 3.587996] systemd[1]: Starting Load Kernel Modules... [ 3.592995] systemd[1]: Starting Journal Service... [ 3.596536] systemd[1]: Listening on udev Control Socket. [ 3.600584] systemd[1]: Reached target Sockets. [ 3.605878] systemd[1]: Starting dracut cmdline hook... [ 3.609750] systemd[1]: Reached target Slices. [ 3.613158] systemd[1]: Reached target Local File Systems. [ 3.619751] systemd[1]: Starting Setup Virtual Console... [ 3.626239] systemd[1]: Starting Create list of required static device nodes for the current kernel... [ 3.634267] systemd[1]: Started Journal Service. [ 3.664954] random: fast init done [ 3.675635] tsc: Refined TSC clocksource calibration: 2399.959 MHz [ 4.218105] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input2 [ 4.255312] libata version 3.00 loaded. [ 4.266384] ata_piix 0000:00:01.1: version 2.13 [ 4.278029] scsi host0: ata_piix [ 4.283712] scsi host1: ata_piix [ 4.286066] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.290432] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 4.498938] ip (320) used greatest stack depth: 13080 bytes left [ 4.587660] ip (343) used greatest stack depth: 12336 bytes left [ 7.846332] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ 8.333854] systemd-journald[101]: Received SIGTERM from PID 1 (systemd). [ 8.640726] SELinux: Disabled at runtime. [ 8.642192] SELinux: Unregistering netfilter hooks [ 8.735432] ip_tables: (C) 2000-2006 Netfilter Core Team [ 8.738293] systemd[1]: Inserted module 'ip_tables' [ 9.521860] systemd-journald[565]: Received request to flush runtime journal from PID 1 [ 9.759306] input: PC Speaker as /devices/platform/pcspkr/input/input3 [ 9.850299] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 10.096806] cryptd: max_cpu_qlen set to 1000 [ 10.165069] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ 10.190018] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 10.268952] AVX version of gcm_enc/dec engaged. [ 10.270756] AES CTR mode by8 optimization enabled [ 10.313722] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni) [ 10.318983] alg: No test for __generic-gcm-aes-aesni (__driver-generic-gcm-aes-aesni) [ 10.538024] EDAC MC: Ver: 3.0.0 [ 10.551266] EDAC sbridge: Seeking for: PCI ID 8086:0ea0 [ 10.551281] EDAC sbridge: Ver: 1.1.2 [ 13.966052] mount.nfs (769) used greatest stack depth: 10704 bytes left [ 24.418749] device-mapper: uevent: version 1.0.3 [ 24.420620] device-mapper: ioctl: 4.37.1-ioctl (2018-04-03) initialised: dm-devel@redhat.com [ 28.566550] libcfs: loading out-of-tree module taints kernel. [ 28.568509] libcfs: module verification failed: signature and/or required key missing - tainting kernel [ 28.593609] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 33.472261] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 33.674781] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 33.678972] alg: No test for adler32 (adler32-zlib) [ 34.567109] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 34.734748] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 34.737166] LNet: Accept secure, port 988 [ 36.281619] Key type lgssc registered [ 36.611291] Lustre: Echo OBD driver; http://www.lustre.org/ [ 39.862238] icp: module license 'CDDL' taints kernel. [ 39.863914] Disabling lock debugging due to kernel taint [ 41.452795] ZFS: Loaded module v2.1.5-1, ZFS pool version 5000, ZFS filesystem version 5 [ 44.693731] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 49.682169] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 51.867360] LDISKFS-fs (vde): file extents enabled, maximum tree depth=5 [ 51.870766] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 54.160529] LDISKFS-fs (vdf): file extents enabled, maximum tree depth=5 [ 54.164336] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 57.493418] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 60.492592] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 60.510667] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 60.517322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 61.587920] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 61.595921] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 61.628380] Lustre: lustre-MDT0000: new disk, initializing [ 61.644400] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 61.650664] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 61.675343] mount.lustre (7188) used greatest stack depth: 10208 bytes left [ 62.375985] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 66.422422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 66.439573] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 66.456949] Lustre: Modifying parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 [ 66.461064] Lustre: 8345:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 66.466386] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 66.467990] Lustre: Skipped 1 previous similar message [ 66.498125] Lustre: lustre-MDT0001: new disk, initializing [ 66.511742] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 66.519070] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 66.523638] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 67.221727] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 69.154332] Lustre: Modifying parameter general.debug_raw_pointers in log params [ 71.412796] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 71.416179] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 71.428913] random: crng init done [ 71.432524] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 71.436196] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 71.506625] Lustre: 9549:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 71.516414] Lustre: lustre-OST0000: new disk, initializing [ 71.519097] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 71.529729] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 72.670047] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 76.858477] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 76.862028] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 76.879272] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 76.882777] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 76.902955] Lustre: 10880:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 76.911592] Lustre: lustre-OST0001: new disk, initializing [ 76.913853] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 76.926164] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 78.146294] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 79.116638] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 79.119951] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 79.125012] Lustre: Skipped 1 previous similar message [ 79.139122] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 79.139150] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 83.160698] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 89.291877] Lustre: Setting parameter general.lod.*.mdt_hash in log params [ 95.494060] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing check_logdir /tmp/testlogs/ [ 96.369732] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing yml_node [ 97.353367] Lustre: DEBUG MARKER: Client: 2.16.53.41 [ 97.969091] Lustre: DEBUG MARKER: MDS: 2.16.53.41 [ 98.591802] Lustre: DEBUG MARKER: OSS: 2.16.53.41 [ 98.973554] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Apr 8 15:28:26 EDT 2025 [ 103.649266] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 104.022061] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 116.595989] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 116.598717] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 116.602659] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 119.188086] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 119.188094] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 119.194738] Lustre: Skipped 3 previous similar messages [ 120.901871] Lustre: server umount lustre-MDT0000 complete [ 121.898165] LustreError: 7196:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744140528 with bad export cookie 10554742949332693319 [ 121.899058] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 121.906316] LustreError: 7196:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 124.195822] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 124.195908] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 124.201860] Lustre: Skipped 3 previous similar messages [ 127.204021] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 127.206832] Lustre: Skipped 2 previous similar messages [ 128.030867] Lustre: server umount lustre-MDT0001 complete [ 135.107438] Lustre: server umount lustre-OST0000 complete [ 142.158142] Lustre: server umount lustre-OST0001 complete [ 143.501571] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 145.181452] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 147.371535] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 148.967054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 150.486962] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 150.489978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 151.995935] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 151.998940] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 153.844127] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 153.865383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 153.944571] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 153.953618] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 153.988989] Lustre: lustre-MDT0000: new disk, initializing [ 154.009081] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 154.014927] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 154.017441] Lustre: Skipped 1 previous similar message [ 154.040999] mount.lustre (17730) used greatest stack depth: 9968 bytes left [ 154.811086] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 158.181051] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 158.204397] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 158.234612] Lustre: 18683:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 158.306519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 158.309714] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 158.331423] mount.lustre (18683) used greatest stack depth: 9504 bytes left [ 159.052228] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 161.526002] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 162.303414] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 163.964645] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 163.967702] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 163.984886] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 163.987743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 164.053818] Lustre: 19882:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 164.065041] Lustre: lustre-OST0000: new disk, initializing [ 164.066358] Lustre: Skipped 1 previous similar message [ 164.068369] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 164.070232] Lustre: Skipped 2 previous similar messages [ 164.089761] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 164.091895] Lustre: Skipped 1 previous similar message [ 165.270618] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 165.874014] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 165.892870] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 167.725566] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 168.483049] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 168.534760] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 169.267658] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 169.313299] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 170.884246] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 170.887387] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 170.891969] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 175.845714] Lustre: server umount lustre-OST0000 complete [ 178.324054] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 178.326511] LustreError: Skipped 1 previous similar message [ 178.327959] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 178.331613] Lustre: Skipped 1 previous similar message [ 180.916069] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 180.917995] Lustre: Skipped 3 previous similar messages [ 182.877757] Lustre: server umount lustre-MDT0000 complete [ 183.816555] LustreError: 17740:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744140590 with bad export cookie 10554742949332694740 [ 183.817738] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 183.824469] LustreError: 17740:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 183.927221] Lustre: server umount lustre-MDT0001 complete [ 185.096304] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 15:29:52 (1744140592) [ 186.695351] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 186.819600] LustreError: 22505:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 186.826869] LustreError: 22505:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 5 previous similar messages [ 186.838131] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 187.575658] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 189.760972] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 189.786798] Lustre: 23369:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 190.616933] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 191.890736] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 192.687568] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 194.367397] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 194.370414] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 195.713863] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 196.880478] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 202.214309] Lustre: Modifying parameter general.osc.*.max_dirty_mb in log params [ 202.216933] Lustre: Skipped 1 previous similar message [ 209.476458] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 209.476851] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 209.481111] Lustre: Skipped 3 previous similar messages [ 210.964145] Lustre: server umount lustre-MDT0000 complete [ 211.872290] LustreError: 25347:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744140618 with bad export cookie 10554742949332695622 [ 211.873896] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 211.879687] LustreError: 25347:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 211.983465] Lustre: server umount lustre-MDT0001 complete [ 222.980101] Lustre: server umount lustre-OST0000 complete [ 227.173352] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 229.301316] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 229.414880] LustreError: 27479:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 229.424615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 229.426392] Lustre: Skipped 2 previous similar messages [ 230.175413] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 232.268764] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 232.293631] Lustre: 28338:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 232.296787] Lustre: 28338:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 1 previous similar message [ 233.109779] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 233.825742] Lustre: Setting parameter general.debug_raw_pointers in log params [ 235.409228] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 235.412612] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 236.623299] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 238.729556] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 238.732194] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 238.746023] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 238.748418] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 238.779695] Lustre: lustre-OST0001: new disk, initializing [ 238.781452] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 239.922801] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 239.922993] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 239.922995] Lustre: Skipped 1 previous similar message [ 239.923184] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 239.943802] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 243.601393] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 257.380163] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 257.384370] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 257.391654] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 257.394091] Lustre: Skipped 2 previous similar messages [ 259.971220] Lustre: server umount lustre-MDT0000 complete [ 260.883397] LustreError: 27463:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744140667 with bad export cookie 10554742949332696469 [ 260.884378] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 260.890565] LustreError: 27463:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 260.998774] Lustre: server umount lustre-MDT0001 complete [ 271.988453] Lustre: server umount lustre-OST0000 complete [ 282.967399] Lustre: server umount lustre-OST0001 complete [ 286.334280] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 288.568025] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 288.688082] LustreError: 1632:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 289.502931] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 291.640607] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 291.665366] Lustre: 2497:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 291.668379] Lustre: 2497:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 2 previous similar messages [ 292.487057] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 293.187046] Lustre: Modifying parameter general.debug_raw_pointers in log params [ 293.188852] Lustre: Skipped 2 previous similar messages [ 294.788789] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 294.791973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 294.879746] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 294.881981] Lustre: Skipped 5 previous similar messages [ 296.064004] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 298.144246] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 298.147137] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 299.355558] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 305.512379] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 316.228830] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 316.232607] Lustre: Skipped 3 previous similar messages [ 318.545625] Lustre: server umount lustre-MDT0000 complete [ 319.528417] LustreError: 1614:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744140726 with bad export cookie 10554742949332697764 [ 319.529980] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 319.536844] LustreError: 1614:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 319.645121] Lustre: server umount lustre-MDT0001 complete [ 330.258019] Lustre: server umount lustre-OST0000 complete [ 341.283515] Lustre: server umount lustre-OST0001 complete [ 343.236766] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 15:32:30 (1744140750) [ 349.547252] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 351.811211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 351.928833] LustreError: 8853:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 352.694898] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 354.892504] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 355.758579] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 358.149508] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 358.153145] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 358.211325] Lustre: 10649:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 358.214996] Lustre: 10649:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 3 previous similar messages [ 359.471655] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 361.253830] LustreError: 10656:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 361.658554] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 361.662927] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 362.966692] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 369.206807] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 370.089069] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 370.090779] Lustre: Skipped 2 previous similar messages [ 380.020101] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 380.023662] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 380.025307] Lustre: Skipped 8 previous similar messages [ 380.724921] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 380.730499] Lustre: Skipped 6 previous similar messages [ 382.061049] Lustre: server umount lustre-MDT0000 complete [ 383.026445] LustreError: 8837:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744140790 with bad export cookie 10554742949332698870 [ 383.027647] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 383.037336] LustreError: 8837:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 383.143449] Lustre: server umount lustre-MDT0001 complete [ 394.754655] Lustre: server umount lustre-OST0000 complete [ 405.759536] Lustre: server umount lustre-OST0001 complete [ 407.716058] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 15:33:35 (1744140815) [ 410.874008] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 413.070756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 413.186670] LustreError: 15166:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 413.947920] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 416.085020] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 416.934846] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 419.377133] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 419.381537] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 420.759972] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 422.971437] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 422.975092] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 423.036461] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 423.039082] Lustre: Skipped 8 previous similar messages [ 424.314319] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 430.593540] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 443.512287] Lustre: server umount lustre-MDT0000 complete [ 444.467213] LustreError: 15150:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744140851 with bad export cookie 10554742949332699976 [ 444.468496] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 444.474318] LustreError: 15150:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 444.934201] LustreError: 15161:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 450.583260] Lustre: server umount lustre-MDT0001 complete [ 452.690008] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 454.076856] LustreError: 3760:0:(client.c:1292:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff88008fe06d80 x1828863807355904/t0(0) o250->MGC192.168.204.136@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 [ 454.955288] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 457.082027] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 457.962317] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 459.196416] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 460.572468] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 1 sec [ 461.322312] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 461.682601] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 468.632907] Lustre: server umount lustre-MDT0000 complete [ 469.659540] Lustre: server umount lustre-MDT0001 complete [ 476.635249] Lustre: server umount lustre-OST0000 complete [ 477.613717] Lustre: server umount lustre-OST0001 complete [ 479.504958] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 15:34:46 (1744140886) [ 482.581139] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 484.807962] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 484.929563] LustreError: 24280:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 484.935299] LustreError: 24280:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 9 previous similar messages [ 485.675909] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 487.795636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 487.822032] Lustre: 25145:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 487.827442] Lustre: 25145:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 5 previous similar messages [ 488.689205] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 491.027869] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 491.032434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 492.336907] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 494.537353] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 494.541793] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 495.847144] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 502.140082] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 502.983104] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 502.985710] Lustre: Skipped 4 previous similar messages [ 512.196576] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 512.197544] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 512.197545] Lustre: Skipped 17 previous similar messages [ 512.203853] Lustre: Skipped 10 previous similar messages [ 517.913306] Lustre: server umount lustre-MDT0000 complete [ 518.889325] LustreError: 25167:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744140925 with bad export cookie 10554742949332701901 [ 518.890588] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 518.890589] LustreError: Skipped 1 previous similar message [ 518.899531] LustreError: 25167:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 519.005854] Lustre: server umount lustre-MDT0001 complete [ 530.233744] Lustre: server umount lustre-OST0000 complete [ 541.284382] Lustre: server umount lustre-OST0001 complete [ 543.140674] LustreError: 29418:0:(class_obd.c:480:class_handle_ioctl()) (null): iocontrol from 'lctl' cmd=400866bb _IOW('f', 187, 8) no device found: rc = -22 [ 543.525167] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 15:35:50 (1744140950) [ 544.102387] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 544.509202] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 15:35:51 (1744140951) [ 546.842363] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 548.365395] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 550.535657] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 552.035741] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 552.039215] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 553.864334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 553.880759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 553.960867] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 553.997715] Lustre: lustre-MDT0000: new disk, initializing [ 554.019098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 554.826680] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 557.350784] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 559.111462] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 559.114766] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 559.131898] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 559.135086] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 560.489410] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 561.160127] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 561.169107] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 563.027390] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 563.507173] Lustre: DEBUG MARKER: create test files [ 571.172083] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 576.943745] Lustre: server umount lustre-OST0000 complete [ 578.047967] Lustre: server umount lustre-MDT0000 complete [ 581.100724] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 581.665769] Key type lgssc unregistered [ 581.738223] LNet: 2286:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 582.743746] LNet: Removed LNI 192.168.204.136@tcp [ 590.563250] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 590.714646] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 590.719536] alg: No test for adler32 (adler32-zlib) [ 591.529476] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 591.580988] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 591.582844] LNet: Accept secure, port 988 [ 593.109637] Key type lgssc registered [ 593.302460] Lustre: Echo OBD driver; http://www.lustre.org/ [ 595.181431] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 595.187861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 596.318275] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 597.103879] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 598.354947] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 600.023533] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 600.028547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 600.093035] Lustre: 5327:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 600.128433] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 601.308004] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 602.509984] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 606.127929] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 606.145095] Lustre: 5335:0:(osd_internal.h:1326:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 606.147931] Lustre: 5335:0:(osd_handler.c:1967:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 606.149858] Lustre: 5335:0:(osd_handler.c:1974:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 606.152136] Lustre: 5335:0:(osd_handler.c:1984:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 606.154702] Lustre: 5335:0:(osd_handler.c:1991:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 606.157035] Lustre: 5335:0:(osd_handler.c:1998:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 610.208568] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 611.238169] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 616.148119] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 616.148254] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 616.148612] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 618.631446] Lustre: server umount lustre-OST0000 complete [ 619.673899] Lustre: server umount lustre-MDT0000 complete [ 622.685569] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 623.215663] Key type lgssc unregistered [ 623.300883] LNet: 7057:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 624.307830] LNet: Removed LNI 192.168.204.136@tcp [ 632.588558] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 632.703764] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 632.730654] alg: No test for adler32 (adler32-zlib) [ 633.537732] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 633.585846] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 633.587749] LNet: Accept secure, port 988 [ 635.113607] Key type lgssc registered [ 635.323109] Lustre: Echo OBD driver; http://www.lustre.org/ [ 637.183886] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 637.190286] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 638.317664] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 639.062315] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 640.288314] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 641.991439] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 641.995002] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 642.055590] Lustre: 10215:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 642.089810] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 643.342129] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 644.557656] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 648.089937] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 648.106149] Lustre: 10222:0:(osd_internal.h:1326:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 648.109712] Lustre: 10222:0:(osd_handler.c:1967:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 648.112611] Lustre: 10222:0:(osd_handler.c:1974:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 648.115203] Lustre: 10222:0:(osd_handler.c:1984:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 648.117738] Lustre: 10222:0:(osd_handler.c:1991:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 648.120506] Lustre: 10222:0:(osd_handler.c:1998:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 652.304824] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 658.100179] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 658.100466] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 658.106814] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 659.966650] Lustre: server umount lustre-OST0000 complete [ 661.067329] Lustre: server umount lustre-MDT0000 complete [ 664.155864] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 664.691709] Key type lgssc unregistered [ 664.779885] LNet: 11876:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 665.784742] LNet: Removed LNI 192.168.204.136@tcp [ 670.097198] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 674.644007] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 674.766141] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 674.779576] alg: No test for adler32 (adler32-zlib) [ 675.595434] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 675.646469] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 675.647975] LNet: Accept secure, port 988 [ 677.173624] Key type lgssc registered [ 677.385227] Lustre: Echo OBD driver; http://www.lustre.org/ [ 679.524265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 681.172905] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 682.681011] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 682.684756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 684.269131] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 684.272559] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 686.006642] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 686.023877] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 686.029493] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 687.100611] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 687.108375] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 687.142588] Lustre: lustre-MDT0000: new disk, initializing [ 687.161111] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 687.168390] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 687.921630] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 691.363953] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 691.380231] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 691.406794] Lustre: Modifying parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 [ 691.413094] Lustre: 16531:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 691.423717] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 691.425348] Lustre: Skipped 1 previous similar message [ 691.458790] Lustre: lustre-MDT0001: new disk, initializing [ 691.479704] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 691.489809] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 691.493688] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 692.258307] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 694.789586] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 695.624736] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 697.339650] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 697.342794] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 697.359904] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 697.362396] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 697.426135] Lustre: 17727:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 697.438666] Lustre: lustre-OST0000: new disk, initializing [ 697.440229] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 697.459280] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 698.667463] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 700.120196] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 700.123412] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 700.145558] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 701.174960] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 701.982940] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 702.031695] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 702.802061] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 702.848708] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 705.140211] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 705.140453] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 705.140968] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 709.387073] Lustre: server umount lustre-OST0000 complete [ 711.508104] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 711.512877] LustreError: Skipped 1 previous similar message [ 711.516134] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 711.524781] Lustre: Skipped 1 previous similar message [ 711.528353] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 711.530389] Lustre: Skipped 1 previous similar message [ 715.156075] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 715.156221] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 715.166530] Lustre: Skipped 1 previous similar message [ 716.411010] Lustre: server umount lustre-MDT0000 complete [ 717.369260] LustreError: 15585:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744141124 with bad export cookie 2224483051324645683 [ 717.370567] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 717.376909] LustreError: 15585:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 717.487110] Lustre: server umount lustre-MDT0001 complete [ 719.536998] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 15:38:46 (1744141126) [ 721.548716] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 724.579550] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 730.547471] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 732.324067] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 734.547675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 736.201235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 737.712379] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 737.715942] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 739.263742] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 739.267098] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 741.019154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 741.034618] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 741.108043] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 741.117708] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 741.152385] Lustre: lustre-MDT0000: new disk, initializing [ 741.168606] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 741.173940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 741.948800] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 745.369494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 745.386450] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 745.408455] Lustre: Modifying parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 [ 745.414933] Lustre: 25818:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 745.427160] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 745.429049] Lustre: Skipped 1 previous similar message [ 745.463823] Lustre: lustre-MDT0001: new disk, initializing [ 745.488417] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 745.497296] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 745.500860] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 746.262821] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 748.750813] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 749.534552] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 751.234702] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 751.238290] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 751.254821] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 751.257526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 751.321902] Lustre: 27013:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 752.587068] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 755.068192] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 755.868104] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 759.163013] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 759.167421] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 759.188500] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 759.971235] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 760.760775] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 760.813074] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 764.180361] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 764.180720] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 764.181080] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 767.371052] Lustre: server umount lustre-OST0000 complete [ 769.188314] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 769.188685] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 769.188686] Lustre: Skipped 1 previous similar message [ 769.194687] Lustre: Skipped 2 previous similar messages [ 774.412578] Lustre: server umount lustre-MDT0000 complete [ 775.373209] LustreError: 28715:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744141182 with bad export cookie 2224483051324646551 [ 775.374914] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 775.379939] LustreError: 28715:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 775.489195] Lustre: server umount lustre-MDT0001 complete [ 777.481879] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 15:39:44 (1744141184) [ 779.167291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 779.314568] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 779.316290] Lustre: Skipped 1 previous similar message [ 780.103167] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 782.273752] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 782.301273] Lustre: 30722:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 783.142578] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 784.461166] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 785.329830] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 787.073773] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 787.076762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 788.376220] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 789.610737] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 790.520062] Lustre: 32750:0:(genops.c:1678:obd_export_evict_by_uuid()) MGS: evicting e3bfa118-a669-4598-9255-31df65ac1a7a at adminstrative request [ 792.163780] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 792.168409] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 802.168139] Lustre: MGS: Client e3bfa118-a669-4598-9255-31df65ac1a7a (at 0@lo) reconnecting [ 802.171112] Lustre: Evicted from MGS (at 192.168.204.136@tcp) after server handle changed from 0x1edef38ca31527fb to 0x1edef38ca3152a71 [ 802.174707] Lustre: MGC192.168.204.136@tcp: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 822.694350] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 822.702353] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 822.727619] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 822.735315] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 822.780859] Lustre: 787:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 822.789896] Lustre: 787:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 1 previous similar message [ 822.805193] Lustre: lustre-OST0001: new disk, initializing [ 822.806773] Lustre: Skipped 1 previous similar message [ 822.808867] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 822.810624] Lustre: Skipped 1 previous similar message [ 822.831522] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 822.834192] Lustre: Skipped 2 previous similar messages [ 824.154939] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 827.101929] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 828.162293] Lustre: server umount lustre-OST0001 complete [ 829.954159] LustreError: 31838:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 829.955306] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 829.955834] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 829.955835] Lustre: Skipped 3 previous similar messages [ 829.974116] LustreError: 31838:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 3 previous similar messages [ 834.980656] LustreError: 31839:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 834.988601] LustreError: 31839:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 1 previous similar message [ 835.424226] Lustre: server umount lustre-OST0000 complete [ 837.444219] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 842.478331] Lustre: server umount lustre-MDT0000 complete [ 843.703256] LustreError: 30737:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744141250 with bad export cookie 2224483051324648049 [ 843.704850] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 843.709269] LustreError: 30737:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 843.816605] Lustre: server umount lustre-MDT0001 complete [ 846.305855] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 15:40:53 (1744141253) [ 846.792622] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 847.372925] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 15:40:54 (1744141254) [ 847.915156] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 848.407714] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 15:40:55 (1744141255) [ 848.776884] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 849.159871] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 15:40:56 (1744141256) [ 849.511582] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 850.651290] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 850.654007] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 850.928250] loop: module loaded [ 850.931388] LDISKFS-fs (loop0): file extents enabled, maximum tree depth=5 [ 850.936310] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 851.275511] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 851.780303] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 852.657513] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 854.722484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 855.699855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 856.645811] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 856.648651] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 857.727455] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 857.732632] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 859.411912] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 859.415205] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 859.430710] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 859.433395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 859.440268] LustreError: 5882:0:(obd_config.c:780:class_setup()) setup lustre-OST0000-osd failed (-28) [ 859.443957] LustreError: 5882:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 859.447323] LustreError: 5882:0:(tgt_mount.c:2231:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 859.450185] LustreError: 5882:0:(super25.c:181:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 863.895839] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 865.619037] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 868.407604] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 870.174725] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 871.872683] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 871.875395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 873.339228] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 873.342468] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 875.096457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 875.113523] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 875.182646] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 875.190553] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 875.223691] Lustre: lustre-MDT0000: new disk, initializing [ 875.240354] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 875.246023] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 876.053241] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 879.806518] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 879.825321] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 879.850700] Lustre: 10352:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 879.919140] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 880.667261] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 883.163639] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 883.931651] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 885.802057] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 885.804875] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 885.824597] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 885.827127] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 887.537431] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 890.328822] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 891.247673] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 891.278921] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 891.303057] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 891.382202] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 892.437949] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 892.479208] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 896.259966] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 896.262025] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 896.265472] Lustre: Skipped 3 previous similar messages [ 896.267413] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 896.268929] Lustre: Skipped 6 previous similar messages [ 899.004092] Lustre: server umount lustre-OST0000 complete [ 906.365111] Lustre: server umount lustre-MDT0000 complete [ 907.345289] LustreError: 9406:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744141314 with bad export cookie 2224483051324648602 [ 907.346735] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 907.351540] LustreError: 9406:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 907.480859] Lustre: server umount lustre-MDT0001 complete [ 909.398277] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 15:41:56 (1744141316) [ 911.118585] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 911.273675] LustreError: 14387:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 912.102436] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 914.333213] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 914.358556] Lustre: 15249:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 914.362518] Lustre: 15249:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 1 previous similar message [ 915.195717] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 916.385039] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 917.136607] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 918.795700] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 918.798860] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 920.212820] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 921.429017] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 922.232561] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 922.281082] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 923.053051] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 925.131402] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 2 sec [ 926.817559] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 926.820308] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 926.839348] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 926.842021] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 926.875842] Lustre: lustre-OST0001: new disk, initializing [ 926.877545] Lustre: Skipped 2 previous similar messages [ 926.879787] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 926.881399] Lustre: Skipped 3 previous similar messages [ 928.076395] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 930.652560] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 931.399006] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 933.134881] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 933.137084] Lustre: Skipped 2 previous similar messages [ 933.138286] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 933.151882] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 933.467540] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 934.176011] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 934.222677] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 942.977675] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 947.357526] Lustre: Failing over lustre-MDT0000 [ 947.468739] Lustre: server umount lustre-MDT0000 complete [ 948.164815] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 948.165302] LustreError: 14404:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 948.172490] Lustre: Skipped 5 previous similar messages [ 953.172510] LustreError: 14404:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 953.178297] LustreError: 14404:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 8 previous similar messages [ 960.917877] LDISKFS-fs (dm-0): recovery complete [ 960.920227] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 960.955628] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 961.027894] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 961.029678] Lustre: Skipped 6 previous similar messages [ 961.040608] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 961.778754] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 965.431042] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 965.431676] Lustre: 14410:0:(ldlm_lib.c:1971:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 966.036077] Lustre: 14387:0:(ldlm_lib.c:1971:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 966.038686] Lustre: 14387:0:(ldlm_lib.c:1971:extend_recovery_timer()) Skipped 1 previous similar message [ 966.041675] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 966.050573] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 966.352611] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 966.758817] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 966.762003] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 1 previous similar message [ 967.063575] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 967.065541] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 1 previous similar message [ 967.068457] Lustre: 21301:0:(ldlm_lib.c:1971:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 967.071221] Lustre: 21301:0:(ldlm_lib.c:1971:extend_recovery_timer()) Skipped 7 previous similar messages [ 967.875101] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 967.877468] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 2 previous similar messages [ 968.179594] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 968.181540] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 2 previous similar messages [ 969.387408] Lustre: 21301:0:(ldlm_lib.c:1971:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 969.391695] Lustre: 21301:0:(ldlm_lib.c:1971:extend_recovery_timer()) Skipped 5 previous similar messages [ 970.194599] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 970.196647] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 4 previous similar messages [ 970.199246] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 970.201654] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 5 previous similar messages [ 973.612678] Lustre: 21301:0:(ldlm_lib.c:1971:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 973.620947] Lustre: 21301:0:(ldlm_lib.c:1971:extend_recovery_timer()) Skipped 10 previous similar messages [ 974.426639] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 974.429781] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 10 previous similar messages [ 974.437631] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 974.443409] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 10 previous similar messages [ 981.961159] Lustre: 21301:0:(ldlm_lib.c:1971:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 981.964048] Lustre: 21301:0:(ldlm_lib.c:1971:extend_recovery_timer()) Skipped 20 previous similar messages [ 982.768606] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 982.770938] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 20 previous similar messages [ 982.773742] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 982.776179] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 20 previous similar messages [ 998.021524] Lustre: 21301:0:(ldlm_lib.c:1971:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 998.027471] Lustre: 21301:0:(ldlm_lib.c:1971:extend_recovery_timer()) Skipped 40 previous similar messages [ 998.830582] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 998.832526] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 40 previous similar messages [ 998.835851] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 998.838287] LustreError: 21301:0:(ldlm_lib.c:2604:replay_request_or_update()) Skipped 40 previous similar messages [ 1024.802582] Lustre: 21301:0:(ldlm_lib.c:2279:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1024.804620] Lustre: 21301:0:(ldlm_lib.c:2289:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1024.806835] Lustre: 21301:0:(genops.c:1508:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 9a2c351b-aae5-4b6c-944b-0f3cdb0a5b98@192.168.204.36@tcp [ 1024.810550] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1024.812073] Lustre: 21301:0:(ldlm_lib.c:1801:abort_req_replay_queue()) @@@ aborted: req@ffff8800a085aa00 x1828864475062912/t0(8589934747) o101->9a2c351b-aae5-4b6c-944b-0f3cdb0a5b98@192.168.204.36@tcp:87/0 lens 592/0 e 0 to 0 dl 1744141442 ref 1 fl Complete:/204/ffffffff rc 0/-1 job:'' uid:0 gid:0 [ 1024.817731] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1024.820422] Lustre: lustre-MDT0000: Denying connection for new client 9a2c351b-aae5-4b6c-944b-0f3cdb0a5b98 (at 192.168.204.36@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 1024.822989] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1024.831724] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1024.831802] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 1024.831811] Lustre: Skipped 2 previous similar messages [ 1024.847313] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:72 to 0x280000401:577) [ 1024.847358] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:39 to 0x2c0000401:545) [ 1029.956067] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1029.957717] LustreError: Skipped 1 previous similar message [ 1029.958627] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1029.961482] Lustre: Skipped 1 previous similar message [ 1029.962976] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1029.964448] Lustre: Skipped 5 previous similar messages [ 1033.436528] Lustre: server umount lustre-OST0000 complete [ 1036.148476] LustreError: 16364:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1036.155916] LustreError: 16364:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 9 previous similar messages [ 1048.378570] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1048.435160] Lustre: server umount lustre-OST0001 complete [ 1049.876148] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1049.879863] LustreError: Skipped 2 previous similar messages [ 1055.417375] Lustre: server umount lustre-MDT0000 complete [ 1056.179890] LustreError: 14386:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1056.185703] LustreError: 14386:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 5 previous similar messages [ 1056.386143] LustreError: 22673:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744141463 with bad export cookie 2224483051324699786 [ 1056.387081] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1056.391479] LustreError: 22673:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1056.491974] Lustre: server umount lustre-MDT0001 complete [ 1058.393555] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 15:44:25 (1744141465) [ 1060.332978] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1060.337314] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1060.340652] Lustre: *** cfs_fail_loc=197, val=0*** [ 1077.407088] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1083.411042] LustreError: 23860:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.136@tcp: failed processing log, type 4: rc = -110 [ 1089.422015] Lustre: *** cfs_fail_loc=197, val=0*** [ 1089.423097] Lustre: Skipped 65 previous similar messages [ 1112.432725] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1113.731457] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1115.178580] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1115.814252] Lustre: server umount lustre-OST0000 complete [ 1118.071766] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 15:45:25 (1744141525) [ 1131.008144] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1131.119097] LustreError: 25786:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1131.833226] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1134.311799] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1136.115802] LustreError: 25818:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1136.118526] LustreError: 25818:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1136.120894] LustreError: 25818:0:(lod_dev.c:506:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 5, retries 0, failed: rc = -5 [ 1136.224058] Lustre: server umount lustre-MDT0000 complete [ 1140.077419] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1140.080442] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1144.751313] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 1147.002187] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1149.257079] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1151.311187] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1153.414527] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1153.423382] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1155.440746] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 1155.450534] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1157.360175] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1157.388961] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1157.473051] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1157.476575] Lustre: Skipped 1 previous similar message [ 1157.484816] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1157.520419] Lustre: lustre-MDT0000: new disk, initializing [ 1157.544201] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1158.655942] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1162.250446] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1162.265322] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1162.294452] Lustre: 31799:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1162.297037] Lustre: 31799:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 2 previous similar messages [ 1162.372611] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1163.136433] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1165.539992] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1166.328095] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1167.958826] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1167.962221] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1167.979170] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1167.982801] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1169.372789] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1171.903954] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1172.651809] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1174.601265] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1174.750689] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1175.742728] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1175.824758] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1179.588908] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1179.589311] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1179.589312] Lustre: Skipped 5 previous similar messages [ 1179.589751] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1179.589753] Lustre: Skipped 13 previous similar messages [ 1179.600805] LustreError: Skipped 1 previous similar message [ 1182.522421] Lustre: server umount lustre-OST0000 complete [ 1189.603604] Lustre: server umount lustre-MDT0000 complete [ 1190.761630] LustreError: 31812:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744141597 with bad export cookie 2224483051324710090 [ 1190.763811] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1190.772080] LustreError: 31812:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1190.927647] Lustre: server umount lustre-MDT0001 complete [ 1193.140947] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 15:46:40 (1744141600) [ 1196.010665] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 1196.615142] Key type lgssc unregistered [ 1196.698860] LNet: 3308:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1197.703786] LNet: Removed LNI 192.168.204.136@tcp [ 1202.834235] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 1207.524294] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1207.725060] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1207.730845] alg: No test for adler32 (adler32-zlib) [ 1208.538124] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 1208.592020] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 1208.593717] LNet: Accept secure, port 988 [ 1210.125679] Key type lgssc registered [ 1210.311935] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1212.195814] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1213.688736] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1215.756385] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1215.762147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1217.890057] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 1217.896091] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1219.803705] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1222.031644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1222.051510] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1222.056530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1223.124370] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1223.130921] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1223.163090] Lustre: lustre-MDT0000: new disk, initializing [ 1223.178564] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1223.183916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1224.315403] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1227.988282] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1227.993946] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1228.019167] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1228.027105] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1228.143744] Lustre: 8265:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1228.166693] Lustre: lustre-OST0000: new disk, initializing [ 1228.169977] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1228.173609] Lustre: Skipped 1 previous similar message [ 1228.205184] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1229.580872] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1230.130135] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1230.134973] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1230.147020] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1233.781181] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 1233.786921] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1233.813800] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 1233.819193] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1233.868677] Lustre: 9336:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1233.887805] Lustre: lustre-OST0001: new disk, initializing [ 1233.891767] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1233.923802] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1235.290372] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1243.940661] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1243.941297] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1243.954823] Lustre: Skipped 1 previous similar message [ 1245.259064] Lustre: server umount lustre-MDT0000 complete [ 1247.034086] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1248.219350] LustreError: 7316:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744141655 with bad export cookie 9160010657227128303 [ 1248.220418] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1248.225975] LustreError: 7316:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1248.248392] Lustre: server umount lustre-OST0000 complete [ 1249.165469] Lustre: server umount lustre-OST0001 complete [ 1251.215219] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 15:47:38 (1744141658) [ 1253.761296] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1255.986803] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1256.215444] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1257.986287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1258.359324] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1260.429684] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 15:47:47 (1744141667) [ 1264.803283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1265.527705] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1266.265031] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1266.983240] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1267.354749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1268.081230] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1273.752706] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 1276.431092] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1279.367089] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1281.627529] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1283.768160] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1283.775147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1286.025887] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 1286.030016] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1288.507470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1288.531402] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1288.651479] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1288.664446] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1288.705227] Lustre: lustre-MDT0000: new disk, initializing [ 1288.733627] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1288.744295] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1289.617599] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1294.049054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1294.072723] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1294.115458] Lustre: Modifying parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 [ 1294.127348] Lustre: 18711:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1294.148155] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1294.151716] Lustre: Skipped 1 previous similar message [ 1294.203514] Lustre: lustre-MDT0001: new disk, initializing [ 1294.238600] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1294.256216] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1294.262586] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1295.412904] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1298.542902] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1299.513061] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1301.970013] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1301.977517] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1302.002061] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1302.008762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1302.111004] Lustre: 19907:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1303.914342] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1306.926393] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1308.034024] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1308.922080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1308.928053] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1308.955987] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1309.104931] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1310.108649] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1310.185768] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1313.956775] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1313.957029] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1313.957669] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1313.957672] Lustre: Skipped 1 previous similar message [ 1313.981579] LustreError: Skipped 1 previous similar message [ 1316.964778] Lustre: server umount lustre-OST0000 complete [ 1318.964271] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1318.967791] Lustre: Skipped 1 previous similar message [ 1318.969290] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1318.971438] Lustre: Skipped 2 previous similar messages [ 1323.972240] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1323.975772] Lustre: Skipped 1 previous similar message [ 1324.437607] Lustre: server umount lustre-MDT0000 complete [ 1325.846645] LustreError: 21587:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744141732 with bad export cookie 9160010657227129437 [ 1325.849603] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1325.858538] LustreError: 21587:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1326.012886] Lustre: server umount lustre-MDT0001 complete [ 1328.899647] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 15:48:56 (1744141736) [ 1331.090423] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1331.288756] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1331.293169] Lustre: Skipped 1 previous similar message [ 1332.420815] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1335.244798] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1335.290842] Lustre: 23601:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1336.539069] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1338.323425] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1339.346595] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1341.800996] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1341.805614] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1343.668969] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1345.289128] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1362.380615] Lustre: *** cfs_fail_loc=159, val=0*** [ 1378.388282] Lustre: lustre-MDT0000: Client 7a0c340b-b7ce-40ed-973d-1b0dba762f90 (at 192.168.204.36@tcp) reconnecting [ 1378.716289] Lustre: *** cfs_fail_loc=159, val=0*** [ 1378.717698] Lustre: Skipped 5 previous similar messages [ 1394.724791] Lustre: lustre-MDT0000: Client 7a0c340b-b7ce-40ed-973d-1b0dba762f90 (at 192.168.204.36@tcp) reconnecting [ 1396.020251] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1396.024348] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1396.030957] Lustre: Skipped 1 previous similar message [ 1396.033606] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1401.469101] Lustre: server umount lustre-OST0000 complete [ 1405.492221] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1405.494813] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1405.499386] Lustre: Skipped 1 previous similar message [ 1405.501154] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1405.503428] Lustre: Skipped 2 previous similar messages [ 1408.634409] Lustre: server umount lustre-MDT0000 complete [ 1409.670427] LustreError: 22723:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744141816 with bad export cookie 9160010657227130305 [ 1409.671745] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1409.677507] LustreError: 22723:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1409.794642] Lustre: server umount lustre-MDT0001 complete [ 1412.728564] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 1413.285130] Key type lgssc unregistered [ 1413.383777] LNet: 26870:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1414.391893] LNet: Removed LNI 192.168.204.136@tcp [ 1421.446789] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1421.566604] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1421.598690] alg: No test for adler32 (adler32-zlib) [ 1422.405153] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 1422.457954] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 1422.460362] LNet: Accept secure, port 988 [ 1423.988637] Key type lgssc registered [ 1424.205228] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1426.307201] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1426.314181] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1427.461243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1428.268112] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1430.473520] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1430.511494] Lustre: 29242:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1430.593749] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1431.331948] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1432.552779] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1433.344845] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1434.952901] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1434.957668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1435.015547] Lustre: 30350:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1435.062518] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1436.351445] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1437.062529] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1437.682238] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1447.545616] Lustre: *** cfs_fail_loc=159, val=0*** [ 1463.554784] Lustre: lustre-MDT0000: Client 0843953c-4227-49cc-b12c-d9e737b52383 (at 192.168.204.36@tcp) reconnecting [ 1463.937799] Lustre: *** cfs_fail_loc=159, val=0*** [ 1463.939789] Lustre: Skipped 1 previous similar message [ 1479.939589] Lustre: lustre-MDT0001: Client 0843953c-4227-49cc-b12c-d9e737b52383 (at 192.168.204.36@tcp) reconnecting [ 1480.199737] Lustre: *** cfs_fail_loc=159, val=0*** [ 1480.201436] Lustre: Skipped 2 previous similar messages [ 1496.201515] Lustre: lustre-MDT0001: Client 0843953c-4227-49cc-b12c-d9e737b52383 (at 192.168.204.36@tcp) reconnecting [ 1496.895516] Lustre: 31708:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1497.492402] Lustre: *** cfs_fail_loc=159, val=0*** [ 1497.493901] Lustre: Skipped 3 previous similar messages [ 1513.497540] Lustre: lustre-MDT0000: Client ceeaea3d-68d9-49e9-b205-c7af079bb023 (at 192.168.204.36@tcp) reconnecting [ 1513.763863] Lustre: *** cfs_fail_loc=159, val=0*** [ 1513.766777] Lustre: Skipped 14 previous similar messages [ 1529.767385] Lustre: lustre-MDT0000: Client ceeaea3d-68d9-49e9-b205-c7af079bb023 (at 192.168.204.36@tcp) reconnecting [ 1530.006421] Lustre: 32061:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1532.212145] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1532.216123] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1532.221735] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1535.220106] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1535.220291] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1535.220301] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1535.230446] Lustre: Skipped 1 previous similar message [ 1536.925876] Lustre: server umount lustre-OST0000 complete [ 1540.228179] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1540.228564] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1540.235657] Lustre: Skipped 1 previous similar message [ 1543.892129] Lustre: server umount lustre-MDT0000 complete [ 1544.882294] LustreError: 28362:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744141951 with bad export cookie 12854983843684226510 [ 1544.883327] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1544.888478] LustreError: 28362:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1544.983430] Lustre: server umount lustre-MDT0001 complete [ 1548.060140] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 1548.598348] Key type lgssc unregistered [ 1548.680791] LNet: 799:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1549.685841] LNet: Removed LNI 192.168.204.136@tcp [ 1556.380292] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1556.530555] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1556.533988] alg: No test for adler32 (adler32-zlib) [ 1557.341542] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 1557.383753] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 1557.386195] LNet: Accept secure, port 988 [ 1558.912618] Key type lgssc registered [ 1559.080758] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1560.892141] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1560.896854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1562.023903] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1562.725882] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1564.717238] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1564.741869] Lustre: 3196:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1564.812435] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1565.521727] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1566.641362] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1567.350655] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1568.855769] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1568.858989] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1568.915530] Lustre: 4305:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1568.947453] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1570.038730] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1571.151224] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1571.948232] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 1573.948771] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 1578.695705] Lustre: 5120:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1579.527216] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight in log params [ 1580.203752] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight in log params [ 1580.205603] Lustre: Skipped 1 previous similar message [ 1580.627703] Lustre: 5522:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1580.632389] Lustre: 5522:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) Skipped 1 previous similar message [ 1581.972087] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1581.974003] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1581.977725] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1583.956088] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1583.956781] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1583.961069] Lustre: Skipped 1 previous similar message [ 1587.292622] Lustre: server umount lustre-OST0000 complete [ 1588.964167] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1588.964658] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1588.971609] Lustre: Skipped 1 previous similar message [ 1593.972011] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1593.973488] Lustre: Skipped 2 previous similar messages [ 1594.250386] Lustre: server umount lustre-MDT0000 complete [ 1595.184884] LustreError: 5457:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744142002 with bad export cookie 10015524726000007580 [ 1595.186600] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1595.199444] LustreError: 5457:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1595.335852] Lustre: server umount lustre-MDT0001 complete [ 1598.315136] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 1598.791295] Key type lgssc unregistered [ 1598.883851] LNet: 6725:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1599.888777] LNet: Removed LNI 192.168.204.136@tcp [ 1605.823767] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1605.960943] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1605.964141] alg: No test for adler32 (adler32-zlib) [ 1606.765424] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 1606.816406] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 1606.817811] LNet: Accept secure, port 988 [ 1608.343627] Key type lgssc registered [ 1608.498841] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1610.110583] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1610.114518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1611.239864] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1611.892694] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1613.829357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1613.854240] Lustre: 9096:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1613.924281] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1614.578708] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1615.647062] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1616.312248] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1617.791037] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1617.793957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1617.848623] Lustre: 10209:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1617.882111] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1618.926547] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1620.019914] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1621.881749] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 1623.882270] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 1627.704651] Lustre: *** cfs_fail_loc=159, val=0*** [ 1643.708342] Lustre: lustre-MDT0000: Client e7012ada-72a8-4be7-a38d-5615b8f3df2d (at 192.168.204.36@tcp) reconnecting [ 1658.933090] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1658.934108] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1658.944945] Lustre: Skipped 1 previous similar message [ 1663.940986] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1663.944457] Lustre: Skipped 2 previous similar messages [ 1664.623832] Lustre: server umount lustre-OST0000 complete [ 1668.948122] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1668.948555] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1668.952299] Lustre: Skipped 1 previous similar message [ 1671.977772] Lustre: server umount lustre-MDT0000 complete [ 1672.823256] LustreError: 8219:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744142079 with bad export cookie 2496523609478398144 [ 1672.825188] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1672.828004] LustreError: 8219:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1672.930247] Lustre: server umount lustre-MDT0001 complete [ 1675.631655] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 1676.098158] Key type lgssc unregistered [ 1676.178792] LNet: 12236:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1677.184000] LNet: Removed LNI 192.168.204.136@tcp [ 1684.644747] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1684.794463] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1684.798436] alg: No test for adler32 (adler32-zlib) [ 1685.614898] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 1685.683719] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 1685.687044] LNet: Accept secure, port 988 [ 1687.230645] Key type lgssc registered [ 1687.396705] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1689.262402] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1689.267459] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1690.393038] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1691.084140] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1693.095959] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1693.121587] Lustre: 14605:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1693.193760] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1693.870971] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1695.000443] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1695.728824] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1697.252933] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1697.255499] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1697.314704] Lustre: 15714:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1697.351888] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1698.435090] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1699.544405] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1700.353041] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 1703.352940] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 1717.263968] Lustre: lustre-OST0000: evicting d0e4ccb7-7741-4391-9db6-116999fec3e2 (at 192.168.204.36@tcp) by administrative request [ 1717.269278] Lustre: Skipped 1 previous similar message [ 1743.412672] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1743.412912] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1743.413565] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1743.427100] Lustre: Skipped 1 previous similar message [ 1747.129417] Lustre: server umount lustre-OST0000 complete [ 1753.268358] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1753.272661] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1753.280061] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1753.283823] Lustre: Skipped 1 previous similar message [ 1754.623306] Lustre: server umount lustre-MDT0000 complete [ 1755.969721] LustreError: 13726:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744142163 with bad export cookie 8333285113547708982 [ 1755.972147] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1755.981470] LustreError: 13726:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1756.120053] Lustre: server umount lustre-MDT0001 complete [ 1759.741470] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 1760.211879] Key type lgssc unregistered [ 1760.289951] LNet: 18250:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1761.294781] LNet: Removed LNI 192.168.204.136@tcp [ 1785.513600] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1785.696057] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1785.699446] alg: No test for adler32 (adler32-zlib) [ 1786.516972] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 1786.584775] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 1786.586409] LNet: Accept secure, port 988 [ 1788.128652] Key type lgssc registered [ 1788.451787] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1791.146824] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1791.155416] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1792.305535] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1793.009643] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1796.015550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1796.072656] Lustre: 22055:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1796.183094] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1797.284086] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1799.084535] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1800.288188] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1802.659394] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1802.665189] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1802.762330] Lustre: 23171:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1802.833466] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1804.578847] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1806.449544] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1806.836559] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 1817.837329] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 1826.868800] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1826.874879] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1826.885010] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1827.860679] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1827.860910] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1827.861744] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1831.171832] Lustre: server umount lustre-OST0000 complete [ 1832.868559] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1832.869088] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1832.869090] Lustre: Skipped 1 previous similar message [ 1832.880943] Lustre: Skipped 1 previous similar message [ 1837.876261] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1837.881574] Lustre: Skipped 2 previous similar messages [ 1838.583931] Lustre: server umount lustre-MDT0000 complete [ 1839.884864] LustreError: 24253:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744142246 with bad export cookie 10041184181652276860 [ 1839.887244] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1839.896623] LustreError: 24253:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1840.042721] Lustre: server umount lustre-MDT0001 complete [ 1843.795740] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 1844.581029] Key type lgssc unregistered [ 1844.687459] LNet: 25048:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1845.696973] LNet: Removed LNI 192.168.204.136@tcp [ 1854.538255] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1854.670341] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1854.674799] alg: No test for adler32 (adler32-zlib) [ 1855.487961] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 1855.550254] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 1855.553079] LNet: Accept secure, port 988 [ 1857.095681] Key type lgssc registered [ 1857.414869] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1859.818360] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1859.826515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1862.492529] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 15:57:49 (1744142269) [ 1867.932011] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 1870.753726] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1873.929342] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1875.574442] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1876.951122] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1876.953821] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1878.396101] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 1878.398602] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1880.317528] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1880.336088] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1880.340836] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1881.418336] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 1881.430924] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1881.475284] Lustre: lustre-MDT0000: new disk, initializing [ 1881.504715] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1881.514639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1882.483968] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1885.922689] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1885.944339] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1885.977034] Lustre: Modifying parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 [ 1885.983545] Lustre: 31008:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1885.996137] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1885.998444] Lustre: Skipped 1 previous similar message [ 1886.031656] Lustre: lustre-MDT0001: new disk, initializing [ 1886.059159] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1886.070362] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1886.077019] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1886.891742] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1889.738666] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1890.734284] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1892.799140] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1892.802791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1892.821823] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1892.825887] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1892.907513] Lustre: 32205:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1892.920955] Lustre: lustre-OST0000: new disk, initializing [ 1892.923076] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1892.946654] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1894.330960] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1896.855498] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1900.238559] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1900.243038] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1900.264384] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1905.252443] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1905.252563] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1905.253215] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1906.527679] Lustre: server umount lustre-OST0000 complete [ 1909.535671] Lustre: server umount lustre-MDT0000 complete [ 1910.260189] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1910.266312] Lustre: Skipped 2 previous similar messages [ 1910.532626] LustreError: 31022:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744142317 with bad export cookie 14402599156732180102 [ 1910.533795] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1910.543398] LustreError: 31022:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1910.668463] Lustre: server umount lustre-MDT0001 complete [ 1913.721677] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 1914.279297] Key type lgssc unregistered [ 1914.365917] LNet: 1908:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1915.375958] LNet: Removed LNI 192.168.204.136@tcp [ 1922.297332] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 1922.422793] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1922.442182] alg: No test for adler32 (adler32-zlib) [ 1923.255605] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 1923.304955] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 1923.307428] LNet: Accept secure, port 988 [ 1924.835745] Key type lgssc registered [ 1925.095306] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1926.931175] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1926.937069] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1928.096358] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1928.884537] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1931.136076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1931.161264] Lustre: 4280:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1931.240588] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1932.019160] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1933.338616] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1934.150798] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1935.890801] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 1935.893722] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1935.950462] Lustre: 5401:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 1935.979722] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1937.227788] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 1937.981571] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 1938.556212] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1945.217007] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1945.223923] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1945.230786] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1945.233504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1945.239661] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 1945.473979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1945.717585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1945.949795] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1945.951292] Lustre: Skipped 2 previous similar messages [ 1945.954588] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1945.957352] Lustre: Skipped 2 previous similar messages [ 1945.958534] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1945.961399] Lustre: Skipped 2 previous similar messages [ 1945.962555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1945.965673] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 1945.967380] Lustre: Skipped 2 previous similar messages [ 1946.200763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1946.443399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1946.676772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1946.903385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1947.133994] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1947.136695] Lustre: Skipped 4 previous similar messages [ 1947.140085] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1947.143188] Lustre: Skipped 4 previous similar messages [ 1947.144335] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1947.146246] Lustre: Skipped 4 previous similar messages [ 1947.147446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1947.150457] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 1947.152186] Lustre: Skipped 4 previous similar messages [ 1947.376023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1947.632190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1947.915199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1948.168236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1948.410790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1948.650582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1948.880570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1949.110240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1949.324153] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1949.325802] Lustre: Skipped 8 previous similar messages [ 1949.329304] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1949.332397] Lustre: Skipped 8 previous similar messages [ 1949.333503] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1949.335156] Lustre: Skipped 8 previous similar messages [ 1949.336198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1949.339353] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 1949.341328] Lustre: Skipped 8 previous similar messages [ 1949.609436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1949.919915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1950.276898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1950.630000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1950.949804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1951.260708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1951.513425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1951.788233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1952.072286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1952.352700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1952.622948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1952.908415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1953.230910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1953.515573] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1953.517751] Lustre: Skipped 13 previous similar messages [ 1953.522021] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1953.524671] Lustre: Skipped 13 previous similar messages [ 1953.525896] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1953.527463] Lustre: Skipped 13 previous similar messages [ 1953.528549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1953.534956] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 1953.539689] Lustre: Skipped 13 previous similar messages [ 1953.808335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1954.065404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1954.307867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1954.561108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1954.833764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1955.108526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1955.365256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1955.653547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1955.931927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1956.188840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1956.443089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1956.666302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1956.901529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1957.134444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1957.364297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1957.589141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1957.812800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1958.033031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1958.266259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1958.553537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1958.796847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1959.033307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1959.265969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1959.507442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1959.743870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1959.975509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1960.201829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1960.426875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1960.652941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1960.873135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1961.097520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1961.329204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1961.543807] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1961.546304] Lustre: Skipped 32 previous similar messages [ 1961.550299] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1961.557230] Lustre: Skipped 32 previous similar messages [ 1961.559123] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1961.561030] Lustre: Skipped 32 previous similar messages [ 1961.562077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1961.564957] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 1961.568976] Lustre: Skipped 32 previous similar messages [ 1961.782396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1962.012072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1962.241615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1962.481053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1962.721328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1962.960823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1963.187722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1963.413302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1963.638016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1963.864136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1964.087032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1964.313992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1964.547079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1964.774514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1965.004160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1965.234930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1965.456745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1965.681226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1965.905963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1966.134884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1966.361158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1966.590147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1966.812521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1967.033641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1967.248204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1967.468364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1967.692028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1967.910229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1968.134073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1968.358431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1968.584522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1968.810745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1969.042092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1969.270739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1969.495710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1969.730531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1969.956521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1970.192908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1970.423863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1970.665144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1970.893659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1971.125529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1971.355010] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1971.585672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1971.815323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1972.050024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1972.276522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1972.498653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1972.728348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1972.958663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1973.194532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1973.432812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1973.657083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1973.890934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1974.114332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1974.355985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1974.581138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1974.809513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1975.052790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1975.284134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1975.517722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1975.748417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1975.982757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1976.212319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1976.444069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1976.675313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1976.902113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1977.126254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1977.353709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1977.571974] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 1977.574684] Lustre: Skipped 69 previous similar messages [ 1977.579586] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1977.586240] Lustre: Skipped 69 previous similar messages [ 1977.588746] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 1977.592504] Lustre: Skipped 69 previous similar messages [ 1977.594876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1977.600248] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 1977.603842] Lustre: Skipped 69 previous similar messages [ 1977.820772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1978.039350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1978.261510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1978.485507] LustreError: 3439:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 1978.488397] LustreError: 3439:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 1978.489988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1978.707189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1978.929006] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1979.147863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1979.371128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1979.598446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1979.822086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1980.038034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1980.254208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1980.469370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1980.692367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1980.914395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1981.138050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1981.363331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1981.577518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1981.799605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1982.014505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1982.229797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1982.456198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1982.684142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1982.912646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1983.146417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1983.377523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1983.608539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1983.844155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1984.082018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1984.309046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1984.538989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1984.769395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1985.004505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1985.237823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1985.457217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1985.686339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1985.911514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1986.150833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1986.382688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1986.607938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1986.828764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1987.048855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1987.265933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1987.491135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1987.708630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1987.935098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1988.157402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1988.387520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1988.615009] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1988.839295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1989.073321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1989.297723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1989.519517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1989.742582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1989.968207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1990.194990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1990.417327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1990.641161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1990.862203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1991.079599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1991.298662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1991.520163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1991.738514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1991.959872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1992.175648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1992.396745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1992.633444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1992.858877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1993.075217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1993.288515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1993.505286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1993.737195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1993.965002] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1994.191226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1994.409857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1994.629710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1994.854176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1995.079534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1995.317697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1995.550466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1995.776529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1995.999320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1996.230270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1996.462283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1996.692935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1996.928519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1997.158112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1997.398523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1997.633824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1997.871295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1998.096836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1998.326869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1998.564528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1998.799617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1999.036159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1999.267185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1999.502974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1999.730500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 1999.954049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2000.174882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2000.397652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2000.620371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2000.849523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2001.082913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2001.324797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2001.551765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2001.776073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2002.007313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2002.238660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2002.465691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2002.693155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2002.914754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2003.137003] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2003.376362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2003.598985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2003.830711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2004.049524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2004.284356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2004.508649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2004.728250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2004.943132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2005.171693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2005.400959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2005.623631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2005.846738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2006.078105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2006.304457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2006.526927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2006.748662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2006.974977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2007.207692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2007.440728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2007.673813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2007.909057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2008.135721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2008.372388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2008.602119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2008.835040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2009.066752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2009.296611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2009.528674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2009.757763] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2009.760740] Lustre: Skipped 141 previous similar messages [ 2009.765800] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2009.768560] Lustre: Skipped 141 previous similar messages [ 2009.769737] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2009.772096] Lustre: Skipped 141 previous similar messages [ 2009.773573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2009.776213] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 2009.779079] Lustre: Skipped 141 previous similar messages [ 2010.006411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2010.244042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2010.483728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2010.720206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2010.947509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2011.178387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2011.435115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2011.678377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2011.917900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2012.158503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2012.393819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2012.632746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2012.862912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2013.099526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2013.332289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2013.567766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2013.807521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2014.048667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2014.279864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2014.525418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2014.758463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2014.987148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2015.206278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2015.430317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2015.668522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2015.904193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2016.140059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2016.370520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2016.600798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2016.831330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2017.064461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2017.308729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2017.538993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2017.769292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2017.995532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2018.225499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2018.464727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2018.699644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2018.932100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2019.159270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2019.388824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2019.619424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2019.861901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2020.102237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2020.335041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2020.588415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2020.828242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2021.066001] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2021.304691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2021.553860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2021.794258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2022.039524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2022.292849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2022.544398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2022.790893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2023.032262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2023.270875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2023.514321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2023.756530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2023.998409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2024.252155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2024.497184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2024.735500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2024.989015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2025.236121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2025.485795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2025.725642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2025.971174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2026.204894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2026.438529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2026.682808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2026.915829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2027.153947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2027.392737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2027.632702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2027.874662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2028.124919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2028.364878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2028.602204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2028.826319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2029.063071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2029.306764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2029.546333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2029.780782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2030.004632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2030.225521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2030.463173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2030.706007] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2030.982460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2031.254833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2031.507034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2031.753911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2032.000388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2032.256898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2032.488522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2032.730675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2032.960188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2033.197109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2033.427949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2033.679658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2033.934543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2034.198756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2034.459455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2034.694935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2034.935286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2035.176675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2035.454801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2035.709782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2035.954935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2036.199540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2036.443256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2036.667261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2036.894131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2037.137441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2037.367376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2037.599813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2037.824019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2038.050124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2038.284885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2038.504870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2038.728535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2038.961457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2039.176930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2039.409948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2039.640436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2039.871472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2040.105291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2040.341520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2040.590067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2040.835191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2041.069070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2041.301935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2041.540360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2041.785023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2042.029499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2042.287409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2042.524726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2042.759014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2043.001135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2043.250524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2043.502381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2043.741972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2043.971930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2044.217072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2044.464525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2044.702667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2044.934917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2045.175840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2045.410010] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2045.641666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2045.876669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2046.113057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2046.360351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2046.613957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2046.855516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2047.109530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2047.346726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2047.588083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2047.842371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2048.092816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2048.332333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2048.581980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2048.834372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2049.090240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2049.334252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2049.584555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2049.829241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2050.073251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2050.327160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2050.590319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2050.848758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2051.111535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2051.368057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2051.618909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2051.872701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2052.115042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2052.354199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2052.593922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2052.843676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2053.088534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2053.322005] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2053.559894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2053.794980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2054.034013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2054.267341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2054.495432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2054.723051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2054.967534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2055.215209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2055.536560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2055.840201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2056.106911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2056.368956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2056.636147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2056.898875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2057.149724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2057.438811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2057.731058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2058.000853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2058.360574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2058.684272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2058.920690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2059.141001] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2059.391530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2059.685543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2060.038548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2060.357259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2060.705371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2060.988239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2061.227048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2061.526547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2061.813097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2062.104215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2062.451314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2062.736548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2063.004171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2063.245819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2063.503549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2063.758539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2064.015535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2064.339572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2064.608756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2064.857701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2065.109014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2065.352880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2065.703240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2066.048551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2066.360533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2066.647613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2066.948794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2067.233182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2067.520891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2067.825174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2068.142544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2068.491896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2068.814314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2069.183366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2069.485908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2069.814098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2070.152970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2070.481884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2070.792341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2071.024767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2071.276130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2071.516522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2071.846940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2072.205452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2072.551553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2072.907072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2073.250395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2073.546335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2073.862868] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2073.866413] Lustre: Skipped 251 previous similar messages [ 2073.873409] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2073.879764] Lustre: Skipped 251 previous similar messages [ 2073.882381] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2073.887510] Lustre: Skipped 251 previous similar messages [ 2073.890412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2073.896776] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 2073.900329] Lustre: Skipped 251 previous similar messages [ 2074.193725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2074.508409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2074.800281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2075.045531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2075.357887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2075.658787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2075.971174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2076.291329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2076.646257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2076.922151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2077.140133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2077.364155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2077.591719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2077.817964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2078.093336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2078.431125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2078.758941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2079.046364] LustreError: 3439:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2079.049057] LustreError: 3439:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2079.054153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2079.291539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2079.582551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2079.904803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2080.178261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2080.431442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2080.676005] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2080.957895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2081.249011] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2081.545518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2081.823542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2082.122070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2082.375197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2082.613846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2082.870399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2083.132484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2083.385512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2083.619067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2083.893838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2084.214983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2084.558190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2084.874821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2085.209106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2085.562100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2085.837878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2086.123295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2086.426345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2086.713738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2087.043995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2087.391135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2087.721133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2087.987547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2088.252339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2088.494372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2088.810008] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2089.159875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2089.512589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2089.860569] LustreError: 3439:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2089.866286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2089.873943] LustreError: 3439:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2090.209108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2090.546299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2090.882520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2091.234891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2091.574415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2091.920926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2092.271560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2092.604016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2092.870034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2093.085528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2093.311696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2093.543212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2093.787969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2094.124565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2094.477024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2094.823328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2095.186593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2095.526002] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2095.871126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2096.180866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2096.480880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2096.756321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2097.074221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2097.391168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2097.621804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2097.850270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2098.085778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2098.353359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2098.676108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2098.993729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2099.262410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2099.493832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2099.779943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2100.071555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2100.363430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2100.662311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2100.934736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2101.226262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2101.541104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2101.756641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2101.991810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2102.277274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2102.535457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2102.828101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2103.095186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2103.320842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2103.550791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2103.764504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2104.004990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2104.303050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2104.639932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2104.917290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2105.164771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2105.440174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2105.757086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2106.088923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2106.358263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2106.614638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2106.866864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2107.196730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2107.491410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2107.707514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2107.938824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2108.155534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2108.374303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2108.637459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2108.951163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2109.192117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2109.409625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2109.617848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2109.846520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2110.154768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2110.498885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2110.842129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2111.167551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2111.429733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2111.656523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2111.886799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2112.184699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2112.456761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2112.683224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2112.899735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2113.186016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2113.504811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2113.843158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2114.192128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2114.454422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2114.665194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2114.872298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2115.114437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2115.441120] LustreError: 3439:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2115.447184] LustreError: 3439:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2115.448387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2115.760467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2116.041925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2116.355942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2116.665317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2116.937530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2117.275770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2117.618924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2117.976091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2118.332281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2118.677240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2119.023467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2119.354384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2119.698583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2120.040160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2120.298788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2120.645519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2120.927560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2121.198722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2121.410262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2121.627033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2121.839516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2122.137464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2122.437859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2122.744745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2123.060125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2123.364718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2123.711947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2124.060570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2124.364547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2124.715735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2125.066168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2125.406339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2125.761131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2126.104136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2126.436454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2126.690909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2126.941726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2127.202402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2127.548413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2127.888742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2128.249015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2128.607161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2128.951972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2129.295002] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2129.633338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2129.972891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2130.233450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2130.570424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2130.916555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2131.227209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2131.492700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2131.714588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2131.930904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2132.162905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2132.509842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2132.802854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2133.131033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2133.456268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2133.791902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2134.138238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2134.492299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2134.774872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2135.123086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2135.495935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2135.772683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2136.017960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2136.233941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2136.450251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2136.716905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2137.063553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2137.387521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2137.739363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2138.034523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2138.341208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2138.681554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2138.935832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2139.201413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2139.545856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2139.904541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2140.207863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2140.452683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2140.663513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2140.875644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2141.113343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2141.383800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2141.659526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2141.924567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2142.266034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2142.609871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2142.935563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2143.207954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2143.475741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2143.755010] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2144.069094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2144.352587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2144.576135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2144.785665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2144.994762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2145.254003] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2145.592821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2145.942859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2146.276409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2146.627232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2146.972864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2147.352537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2147.691552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2147.987439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2148.269107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2148.536816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2148.858312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2149.203442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2149.515209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2149.846269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2150.057804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2150.262959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2150.480707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2150.724839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2151.043558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2151.383791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2151.710938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2152.037550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2152.372082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2152.720539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2153.058910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2153.394563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2153.737060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2154.067192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2154.409396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2154.745126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2155.106805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2155.453270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2155.797302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2156.136523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2156.461050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2156.796066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2157.135555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2157.466940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2157.804853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2158.094932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2158.413163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2158.759803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2159.101785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2159.435962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2159.791549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2160.139139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2160.434952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2160.783380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2161.125552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2161.411922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2161.706529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2162.015321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2162.287251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2162.616543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2162.940549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2163.249680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2163.492886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2163.799232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2164.068928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2164.403116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2164.668083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2165.014844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2165.361402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2165.692126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2165.966797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2166.229124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2166.525555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2166.832292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2167.148308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2167.499908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2167.844934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2168.080958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2168.315634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2168.558274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2168.860030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2169.143206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2169.394790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2169.634023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2169.873022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2170.143954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2170.456561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2170.762403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2171.112040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2171.489174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2171.857382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2172.126347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2172.348740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2172.564697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2172.795527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2173.078983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2173.355969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2173.700737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2174.046080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2174.343575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2174.684198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2175.031433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2175.394707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2175.729821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2176.076171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2176.434580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2176.739923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2177.056773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2177.348520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2177.596529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2177.925182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2178.188093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2178.403461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2178.635460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2178.907768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2179.215310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2179.453383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2179.740067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2180.065063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2180.382096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2180.703558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2181.082774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2181.447332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2181.773946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2182.105427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2182.448889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2182.695957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2182.979995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2183.322062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2183.637352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2183.877905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2184.180552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2184.545420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2184.884159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2185.199763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2185.495831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2185.846171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2186.202256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2186.546759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2186.852297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2187.151893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2187.484286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2187.725716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2187.972284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2188.210301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2188.477244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2188.819332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2189.162422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2189.494897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2189.806793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2190.117184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2190.450906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2190.811309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2191.152789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2191.419815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2191.687147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2191.994529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2192.292998] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2192.569281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2192.937197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2193.288737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2193.607898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2193.970546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2194.299243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2194.566025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2194.900060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2195.276898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2195.621186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2195.978252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2196.361558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2196.630102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2196.855853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2197.076517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2197.296022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2197.551415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2197.896577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2198.212629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2198.562851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2198.896888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2199.189231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2199.550810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2199.901737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2200.177537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2200.466568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2200.782802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2201.154309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2201.466309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2201.813244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2202.158208] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2202.162509] Lustre: Skipped 427 previous similar messages [ 2202.170868] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2202.177663] Lustre: Skipped 427 previous similar messages [ 2202.180308] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2202.184229] Lustre: Skipped 427 previous similar messages [ 2202.186829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2202.192912] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 2202.197744] Lustre: Skipped 427 previous similar messages [ 2202.498189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2202.735714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2202.981772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2203.280411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2203.596684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2203.886375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2204.197805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2204.557257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2204.838912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2205.052520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2205.261466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2205.488806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2205.816705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2206.084913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2206.419839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2206.746384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2207.044967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2207.256290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2207.474711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2207.695686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2208.015234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2208.318551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2208.630816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2208.952459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2209.285112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2209.513663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2209.723129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2209.933671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2210.183024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2210.416968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2210.714738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2211.011246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2211.267958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2211.486933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2211.709293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2211.992154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2212.305523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2212.604871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2212.965599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2213.329058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2213.658980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2214.020363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2214.388570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2360.697461] LustreError: 5408:0:(ofd_dev.c:1668:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 2360.704257] LustreError: 3439:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 2360.710402] LustreError: 3439:0:(osp_precreate.c:654:osp_precreate_send()) Skipped 1 previous similar message [ 2465.412005] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2465.413800] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2465.416426] Lustre: Skipped 43 previous similar messages [ 2465.417847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2466.820568] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2466.821426] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2469.287077] Lustre: server umount lustre-OST0000 complete [ 2471.829220] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2471.832716] Lustre: Skipped 2 previous similar messages [ 2476.571096] Lustre: server umount lustre-MDT0000 complete [ 2476.836332] LustreError: 3420:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2476.845279] LustreError: 3420:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 3 previous similar messages [ 2477.819721] LustreError: 5426:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744142884 with bad export cookie 4396371994836539950 [ 2477.821187] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2477.827433] LustreError: 5426:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2477.974993] Lustre: server umount lustre-MDT0001 complete [ 2481.156505] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 2481.954922] Key type lgssc unregistered [ 2482.062287] LNet: 10666:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2483.070046] LNet: Removed LNI 192.168.204.136@tcp [ 2490.705804] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 2490.861024] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2490.879692] alg: No test for adler32 (adler32-zlib) [ 2491.684934] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 2491.725497] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 2491.728138] LNet: Accept secure, port 988 [ 2493.253627] Key type lgssc registered [ 2493.432538] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2496.061567] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2496.070400] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2497.258114] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2498.315308] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2501.368585] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2501.416281] Lustre: 13035:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 2501.524892] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2502.612294] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2504.319138] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2505.510262] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2507.401123] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2507.407653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2507.504894] Lustre: 14145:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 2507.566057] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2508.892403] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2510.629345] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2511.567429] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 2514.437375] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2515.028774] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2515.037147] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2515.041871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.048451] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 2515.048468] LustreError: 12194:0:(osp_precreate.c:1266:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 2516.580315] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2516.580410] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2516.580985] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2517.789211] Lustre: server umount lustre-OST0000 complete [ 2535.147655] LustreError: 12194:0:(osp_precreate.c:1266:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 2545.489036] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2545.495096] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2545.593887] Lustre: 15762:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 2545.649444] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2547.394328] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2549.285467] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2556.646472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.652831] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.652880] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.204.136@tcp (at 0@lo) [ 2560.076964] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 10 sec [ 2561.296966] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a92ea000.ost_server_uuid 50 [ 2561.844448] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8800a92ea000.ost_server_uuid in FULL state after 0 sec [ 2566.660438] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2566.660986] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2566.660987] Lustre: Skipped 1 previous similar message [ 2566.666787] Lustre: Skipped 2 previous similar messages [ 2568.878104] Lustre: server umount lustre-OST0000 complete [ 2571.604610] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2571.609811] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2571.619767] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2571.623424] Lustre: Skipped 1 previous similar message [ 2576.374430] Lustre: server umount lustre-MDT0000 complete [ 2576.676195] LustreError: 12175:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2576.684799] LustreError: 12175:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 3 previous similar messages [ 2577.728714] LustreError: 12158:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744142984 with bad export cookie 15379493409481191844 [ 2577.730997] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2577.739330] LustreError: 12158:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2577.873252] Lustre: server umount lustre-MDT0001 complete [ 2581.691941] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 2582.482067] Key type lgssc unregistered [ 2582.588115] LNet: 17888:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2583.596140] LNet: Removed LNI 192.168.204.136@tcp [ 2594.614915] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 2594.779826] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2594.793697] alg: No test for adler32 (adler32-zlib) [ 2595.612061] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 2595.684800] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 2595.687734] LNet: Accept secure, port 988 [ 2597.230646] Key type lgssc registered [ 2597.546844] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2600.669340] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2600.678026] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2601.865260] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2602.991804] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2606.054924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2606.101532] Lustre: 20671:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 2606.214866] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2607.367412] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2608.392026] Lustre: Modifying parameter general.debug_raw_pointers in log params [ 2610.765159] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2610.770971] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2610.866139] Lustre: 21595:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 2610.916746] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2611.919376] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 2612.594460] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2615.756772] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2615.761506] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2615.783312] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2615.788803] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2615.839548] Lustre: 22594:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 2615.861170] Lustre: lustre-OST0001: new disk, initializing [ 2615.864691] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2615.903380] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2617.749081] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2621.422285] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2621.424382] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2621.441150] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2624.258927] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2628.998081] Lustre: Setting parameter general.lod.*.mdt_hash in log params [ 2641.461344] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2641.461950] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2641.476135] Lustre: Skipped 1 previous similar message [ 2646.244816] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2646.251049] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2646.260928] Lustre: Skipped 1 previous similar message [ 2646.265067] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2646.268996] Lustre: Skipped 2 previous similar messages [ 2647.348316] Lustre: server umount lustre-MDT0000 complete [ 2648.736022] LustreError: 19792:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744143055 with bad export cookie 10874451908839025746 [ 2648.738775] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2648.748506] LustreError: 19792:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2648.902129] Lustre: server umount lustre-MDT0001 complete [ 2660.507552] Lustre: server umount lustre-OST0000 complete [ 2671.947124] Lustre: server umount lustre-OST0001 complete [ 2673.070535] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 2673.094647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2673.493597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 2673.522392] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2673.920638] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2673.926374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 2673.953331] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2673.958993] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2674.340159] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2674.343193] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 2674.365416] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2674.368259] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2678.598678] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 2681.816901] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2682.013517] LustreError: 26491:0:(ldlm_lib.c:1095:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2682.023249] LustreError: 26491:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 2 previous similar messages [ 2682.041016] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2683.122029] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2686.209445] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2686.253246] Lustre: 27357:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for mylustre-sptlrpc, check MGS config [ 2687.472276] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2688.504822] Lustre: Modifying parameter general.debug_raw_pointers in log params [ 2690.838059] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2690.843755] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2690.937081] Lustre: 28284:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for mylustre-sptlrpc, check MGS config [ 2690.987980] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2690.991839] Lustre: Skipped 1 previous similar message [ 2691.985835] LustreError: 28521:0:(ldlm_lib.c:1095:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2692.001157] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49761) [ 2692.736567] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2695.752832] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2695.758376] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2697.591079] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2700.789154] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2702.019103] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 2711.364508] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2711.368929] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2711.376541] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2711.380323] Lustre: Skipped 3 previous similar messages [ 2716.885622] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2716.889147] Lustre: Skipped 7 previous similar messages [ 2717.393561] Lustre: server umount mylustre-MDT0000 complete [ 2718.698919] LustreError: 26469:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744143125 with bad export cookie 10874451908839028539 [ 2718.701200] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2718.710876] LustreError: 26469:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2718.848346] Lustre: server umount mylustre-MDT0001 complete [ 2730.918933] Lustre: server umount mylustre-OST0000 complete [ 2742.311276] Lustre: server umount mylustre-OST0001 complete [ 2743.425098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 2743.454911] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2743.841693] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 2743.869172] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2744.262579] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2744.268555] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 2744.292183] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2744.298156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2744.689961] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2744.699387] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 2744.725304] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2744.730865] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2748.957903] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 2752.085820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2752.292698] LustreError: 586:0:(ldlm_lib.c:1095:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2752.320365] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2752.324005] Lustre: Skipped 1 previous similar message [ 2753.457911] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2756.503867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2756.550130] Lustre: 1480:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for tfs-sptlrpc, check MGS config [ 2756.556086] Lustre: 1480:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 1 previous similar message [ 2757.814508] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2758.868340] Lustre: Modifying parameter general.debug_raw_pointers in log params [ 2761.256108] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2761.262109] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2762.402776] LustreError: 2415:0:(ldlm_lib.c:1095:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2762.403877] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49793) [ 2762.641909] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2765.674572] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2765.680755] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2767.504554] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2770.759875] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2772.018260] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 2781.684523] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2781.688659] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2781.695084] Lustre: Skipped 3 previous similar messages [ 2781.698185] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 2785.117709] Lustre: server umount tfs-MDT0000 complete [ 2786.443066] LustreError: 563:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744143193 with bad export cookie 10874451908839030205 [ 2786.445445] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2786.454971] LustreError: 563:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2786.597784] Lustre: server umount tfs-MDT0001 complete [ 2798.023393] Lustre: server umount tfs-OST0000 complete [ 2809.427196] Lustre: server umount tfs-OST0001 complete [ 2810.570901] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 2810.599036] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2810.994519] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 2811.024207] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2811.418679] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2811.427344] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 2811.453535] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2811.458787] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2811.847544] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2811.854393] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 2811.878065] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2811.884951] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2816.248921] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 2819.398001] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2819.580430] LustreError: 7102:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2819.601739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2819.605629] Lustre: Skipped 3 previous similar messages [ 2820.751748] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2823.842362] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2823.891001] Lustre: 7965:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 2823.897120] Lustre: 7965:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 2 previous similar messages [ 2825.145322] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2826.188960] Lustre: Modifying parameter general.debug_raw_pointers in log params [ 2828.472305] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2828.478152] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2829.613579] LustreError: 9131:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2829.616508] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49825) [ 2830.350637] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2833.376456] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2833.382622] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2835.239112] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2838.430721] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2846.939108] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 16:14:14 (1744143254) [ 2859.044644] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2859.050725] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2859.058073] Lustre: Skipped 3 previous similar messages [ 2859.061473] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2859.065221] Lustre: Skipped 3 previous similar messages [ 2864.280137] Lustre: server umount lustre-MDT0000 complete [ 2864.516302] LustreError: 7101:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2864.525081] LustreError: 7101:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 3 previous similar messages [ 2865.568886] LustreError: 7080:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744143272 with bad export cookie 10874451908839031647 [ 2865.571032] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2865.580513] LustreError: 7080:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2869.524772] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2869.531444] Lustre: Skipped 4 previous similar messages [ 2871.733758] Lustre: server umount lustre-MDT0001 complete [ 2879.119594] Lustre: server umount lustre-OST0000 complete [ 2886.520329] Lustre: server umount lustre-OST0001 complete [ 2888.350205] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 2890.900761] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 2893.902987] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2896.143780] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2898.259598] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2898.265159] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2900.487299] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2900.492782] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2902.903670] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2902.928383] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 2902.937674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 2903.036714] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 2903.039443] Lustre: Skipped 1 previous similar message [ 2903.047448] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2903.085869] Lustre: lustre-MDT0000: new disk, initializing [ 2903.113256] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2903.116946] Lustre: Skipped 3 previous similar messages [ 2903.125224] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2904.275193] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2908.648508] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2908.676084] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2908.726934] Lustre: 15641:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 2908.736364] Lustre: 15641:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 2 previous similar messages [ 2908.755152] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2908.758602] Lustre: Skipped 1 previous similar message [ 2908.818034] Lustre: lustre-MDT0001: new disk, initializing [ 2908.876026] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2908.888827] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2910.071057] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2914.509518] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2914.516364] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2914.534574] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2914.541479] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2914.650137] Lustre: lustre-OST0000: new disk, initializing [ 2914.654093] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2916.467674] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2916.593946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2916.599793] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2916.625715] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2920.819894] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2920.825758] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2920.851429] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2920.856914] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2920.918759] Lustre: lustre-OST0001: new disk, initializing [ 2920.921915] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2922.407006] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2922.412218] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2922.441775] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2922.741822] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2927.122392] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 16:15:34 (1744143334) [ 2930.410451] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 16:15:37 (1744143337) [ 2941.652350] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2941.652653] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2941.653334] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2941.653336] Lustre: Skipped 7 previous similar messages [ 2941.668745] LustreError: Skipped 1 previous similar message [ 2947.355001] Lustre: server umount lustre-OST0000 complete [ 2947.461188] LustreError: 16591:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2947.470014] LustreError: 16591:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 5 previous similar messages [ 2954.796136] Lustre: server umount lustre-MDT0000 complete [ 2956.123039] LustreError: 16605:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744143363 with bad export cookie 10874451908839032977 [ 2956.125294] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2956.135628] LustreError: 16605:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2962.301208] Lustre: server umount lustre-MDT0001 complete [ 2965.996805] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 2975.991014] Lustre: server umount lustre-OST0001 complete [ 2977.931833] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 2983.107272] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 2985.130234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2986.624787] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2988.021308] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 2988.025353] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2989.645290] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 2989.649038] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2991.542771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2991.569388] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2991.652492] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 2991.655681] Lustre: Skipped 1 previous similar message [ 2991.667558] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2991.708511] Lustre: lustre-MDT0000: new disk, initializing [ 2991.745678] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2992.771644] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 2997.183056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2997.207023] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2997.255294] Lustre: 24179:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 2997.262061] Lustre: 24179:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 2 previous similar messages [ 2997.370258] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2998.670095] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3001.992189] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3003.224767] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3007.197408] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3007.211088] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3007.284844] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3007.303833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3007.793803] Lustre: lustre-OST0000: new disk, initializing [ 3007.801335] Lustre: Skipped 1 previous similar message [ 3007.814025] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3007.826587] Lustre: Skipped 2 previous similar messages [ 3008.815650] hrtimer: interrupt took 4513333 ns [ 3009.482068] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3009.485767] Lustre: Skipped 1 previous similar message [ 3009.522582] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3009.740304] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3014.089444] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3022.181103] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3029.653672] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3029.656436] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3029.656438] Lustre: Skipped 9 previous similar messages [ 3029.758735] Lustre: Skipped 6 previous similar messages [ 3034.626803] Lustre: server umount lustre-OST0000 complete [ 3037.444249] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3037.447139] LustreError: Skipped 1 previous similar message [ 3043.615783] Lustre: server umount lustre-MDT0000 complete [ 3044.676456] LustreError: 24557:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3044.684198] LustreError: 24557:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 4 previous similar messages [ 3045.278697] LustreError: 23229:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744143452 with bad export cookie 10874451908839034447 [ 3045.280158] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3045.288263] LustreError: 23229:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3045.480809] Lustre: server umount lustre-MDT0001 complete [ 3050.425966] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 3051.917891] Key type lgssc unregistered [ 3052.070028] LNet: 27334:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3053.077299] LNet: Removed LNI 192.168.204.136@tcp [ 3066.574920] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3066.901638] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3066.912670] alg: No test for adler32 (adler32-zlib) [ 3067.794735] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 3067.897845] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 3067.899631] LNet: Accept secure, port 988 [ 3069.458630] Key type lgssc registered [ 3069.911709] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3073.441724] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3073.457217] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3074.791459] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3076.357655] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3081.032072] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3081.110537] Lustre: 29775:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3081.327901] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3083.344088] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3086.821705] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3089.088582] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3094.044191] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3094.071923] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3094.275072] Lustre: 30894:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3094.375809] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3097.541099] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3099.388855] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3100.969339] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3101.800120] Lustre: Setting parameter lustre-OST0000.ost.unknown_param in log lustre-OST0000 [ 3104.391754] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3104.405855] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3104.418941] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3106.408117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3109.196219] Lustre: server umount lustre-OST0000 complete [ 3116.342599] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3116.347837] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3116.357407] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3116.360397] Lustre: Skipped 1 previous similar message [ 3117.654800] Lustre: server umount lustre-MDT0000 complete [ 3119.525005] LustreError: 28896:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744143526 with bad export cookie 3759158746153982599 [ 3119.526041] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3119.532050] LustreError: 28896:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3119.710316] Lustre: server umount lustre-MDT0001 complete [ 3124.281967] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 3125.599309] Key type lgssc unregistered [ 3125.749656] LNet: 364:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3126.757693] LNet: Removed LNI 192.168.204.136@tcp [ 3134.573200] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3134.837750] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3134.847807] alg: No test for adler32 (adler32-zlib) [ 3135.698553] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 3135.767290] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 3135.769933] LNet: Accept secure, port 988 [ 3137.314617] Key type lgssc registered [ 3137.865478] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3142.119874] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3142.128371] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3142.135673] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3159.276262] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3165.279646] LustreError: 1629:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.136@tcp: failed processing log, type 4: rc = -110 [ 3195.348824] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3198.278710] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3201.589186] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3202.778752] Lustre: server umount lustre-OST0000 complete [ 3211.172865] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 3212.662968] Key type lgssc unregistered [ 3212.806950] LNet: 3401:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3213.822059] LNet: Removed LNI 192.168.204.136@tcp [ 3232.583390] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3232.731247] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3232.765538] alg: No test for adler32 (adler32-zlib) [ 3233.630240] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 3233.711095] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 3233.714174] LNet: Accept secure, port 988 [ 3235.258284] Key type lgssc registered [ 3235.512814] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3239.009014] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3239.387434] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3239.706064] LDISKFS-fs (loop0): file extents enabled, maximum tree depth=5 [ 3239.710113] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3240.039389] LDISKFS-fs (loop0): file extents enabled, maximum tree depth=5 [ 3240.041921] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3240.306732] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3241.433870] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3242.478669] LDISKFS-fs (loop0): file extents enabled, maximum tree depth=5 [ 3242.481087] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3243.543792] LDISKFS-fs (loop0): file extents enabled, maximum tree depth=5 [ 3243.546874] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3244.545101] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3244.558765] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3244.562700] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3244.578024] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3244.581159] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 3244.584208] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3244.588735] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3244.592972] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3244.597124] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3244.600278] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 3244.603088] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 3244.606970] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3244.611997] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3244.615004] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3244.619827] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3244.623954] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3244.627252] Lustre: 7552:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3246.906836] Lustre: server umount MGS complete [ 3247.203558] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3247.333006] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3247.347045] Lustre: lustre-MDT0000: Aborting client recovery [ 3247.611704] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3247.628737] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3247.643323] Lustre: 8088:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3247.648263] Lustre: 8088:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3247.652997] Lustre: 8088:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3247.657126] Lustre: 8088:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 3247.661048] Lustre: 8088:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3247.666807] Lustre: 8088:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 3247.671850] Lustre: 8088:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3247.680886] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3247.685811] Lustre: 8088:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3248.032864] LDISKFS-fs (loop2): file extents enabled, maximum tree depth=5 [ 3248.036518] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3248.049655] LDISKFS-fs (loop2): file extents enabled, maximum tree depth=5 [ 3248.053399] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3248.065005] Lustre: 8186:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3248.068729] Lustre: 8186:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3248.072036] Lustre: 8186:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3248.075377] Lustre: 8186:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3248.081139] Lustre: 8186:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3248.083693] Lustre: 8186:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3248.087325] Lustre: 8186:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3248.153693] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3248.156086] Lustre: Skipped 1 previous similar message [ 3248.160382] Lustre: lustre-OST0000: Aborting recovery [ 3248.403193] LDISKFS-fs (loop3): file extents enabled, maximum tree depth=5 [ 3248.406327] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3248.423003] LDISKFS-fs (loop3): file extents enabled, maximum tree depth=5 [ 3248.425284] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3248.434954] Lustre: 8280:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3248.439722] Lustre: 8280:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3248.444542] Lustre: 8280:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3248.447399] Lustre: 8280:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3248.449835] Lustre: 8280:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3248.452452] Lustre: 8280:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3248.455155] Lustre: 8280:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3248.459506] Lustre: Found index 1 for lustre-OST0001, updating log [ 3248.460677] Lustre: Skipped 1 previous similar message [ 3248.463515] Lustre: 8280:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3248.466460] Lustre: 8280:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 1 previous similar message [ 3257.486187] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3257.491044] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3257.493753] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 3257.495142] LustreError: 8016:0:(lod_dev.c:506:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 3257.496169] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 3257.496313] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3257.509048] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3259.459176] LustreError: 7986:0:(mdt_identity.c:95:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 180us: rc = -2 [ 3272.517146] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3272.517317] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3272.525347] Lustre: Skipped 3 previous similar messages [ 3276.002560] Lustre: server umount lustre-MDT0000 complete [ 3276.258141] LustreError: 7970:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744143683 with bad export cookie 2692129360949217407 [ 3276.259079] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3276.266507] LustreError: 7970:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3277.524358] LustreError: 7984:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3277.524468] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3277.524711] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3277.524712] Lustre: Skipped 3 previous similar messages [ 3277.545880] LustreError: 7984:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 3 previous similar messages [ 3281.531877] LustreError: 8589:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3281.531879] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3281.531884] Lustre: Skipped 1 previous similar message [ 3281.540230] LustreError: 8589:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 1 previous similar message [ 3282.384926] Lustre: server umount lustre-MDT0001 complete [ 3288.665296] Lustre: server umount lustre-OST0000 complete [ 3295.067549] Lustre: server umount lustre-OST0001 complete [ 3297.404697] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 16:21:44 (1744143704) [ 3306.227029] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 3308.587932] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3311.183238] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3313.423381] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3315.387064] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3315.398000] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3317.568270] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 3317.574647] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3319.627962] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3319.651641] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3319.729104] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 3319.739351] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3319.774996] Lustre: lustre-MDT0000: new disk, initializing [ 3319.799806] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3319.802192] Lustre: Skipped 1 previous similar message [ 3319.807469] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3320.701252] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3324.639846] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3324.658520] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3324.685661] Lustre: Modifying parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 [ 3324.694001] Lustre: 14331:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3324.707591] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3324.709890] Lustre: Skipped 1 previous similar message [ 3324.743711] Lustre: lustre-MDT0001: new disk, initializing [ 3324.779365] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3324.793678] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3324.797147] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3325.702940] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3329.193369] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3330.787252] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3333.815971] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3333.826072] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3333.860551] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3333.866374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3333.968065] Lustre: 15528:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3333.994355] Lustre: lustre-OST0000: new disk, initializing [ 3333.996397] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3334.065752] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3336.440823] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3337.275577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3337.279747] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3337.317430] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3339.711710] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3341.953910] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff in log lustre-MDT0000 [ 3345.504901] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff in log lustre-MDT0000 [ 3354.300550] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb in log lustre-client [ 3402.404321] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3402.404492] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3402.404755] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3402.404756] Lustre: Skipped 1 previous similar message [ 3402.431258] Lustre: Skipped 2 previous similar messages [ 3405.861823] Lustre: server umount lustre-OST0000 complete [ 3407.412010] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3407.412270] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3407.412271] Lustre: Skipped 1 previous similar message [ 3407.419133] Lustre: Skipped 1 previous similar message [ 3412.888840] Lustre: server umount lustre-MDT0000 complete [ 3413.867830] LustreError: 13385:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744143820 with bad export cookie 2692129360949220109 [ 3413.869059] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3413.873170] LustreError: 13385:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3413.985514] Lustre: server umount lustre-MDT0001 complete [ 3416.152091] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3416.956080] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3418.280079] Lustre: server umount MGS complete [ 3421.463582] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3421.619767] LustreError: 20511:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3421.627626] LustreError: 20523:0:(qmt_pool.c:1513:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3421.640541] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3422.659078] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3424.999056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3425.027492] Lustre: 21375:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3425.895058] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3427.152836] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3428.037661] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3430.160521] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3430.165970] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3430.282827] mount.lustre (22487) used greatest stack depth: 9328 bytes left [ 3431.717414] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3433.117751] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3446.308616] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3446.313959] LustreError: Skipped 1 previous similar message [ 3446.317636] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3446.326293] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3446.330541] Lustre: Skipped 3 previous similar messages [ 3450.293005] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3451.510512] Lustre: server umount lustre-OST0000 complete [ 3455.124532] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3455.128997] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3455.135686] Lustre: Skipped 1 previous similar message [ 3458.999445] Lustre: server umount lustre-MDT0000 complete [ 3460.308557] LustreError: 20528:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3460.318355] LustreError: 20528:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 1 previous similar message [ 3460.388780] LustreError: 20494:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744143867 with bad export cookie 2692129360949221936 [ 3460.390791] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3460.400459] LustreError: 20494:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3460.553717] Lustre: server umount lustre-MDT0001 complete [ 3464.321349] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 3465.097419] Key type lgssc unregistered [ 3465.190858] LNet: 24895:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3466.197099] LNet: Removed LNI 192.168.204.136@tcp [ 3478.660935] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 3484.368971] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3484.593538] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3484.598623] alg: No test for adler32 (adler32-zlib) [ 3485.416554] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 3485.487971] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 3485.490975] LNet: Accept secure, port 988 [ 3487.034750] Key type lgssc registered [ 3487.362803] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3490.343098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3492.589726] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3494.665794] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3494.673678] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3496.937510] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 3496.942380] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3499.107009] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3499.126226] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3499.135545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3500.219618] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 3500.231799] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3500.273873] Lustre: lustre-MDT0000: new disk, initializing [ 3500.292693] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3500.300976] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3501.264083] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3505.623901] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3505.651077] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3505.694206] Lustre: Modifying parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 [ 3505.707017] Lustre: 30628:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3505.730407] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3505.733828] Lustre: Skipped 1 previous similar message [ 3505.782565] Lustre: lustre-MDT0001: new disk, initializing [ 3505.818102] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3505.835566] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3505.842788] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3506.955866] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3510.106021] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3511.240791] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3513.552066] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3513.554815] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3513.569494] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3513.571965] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3513.637947] Lustre: 31828:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3513.647704] Lustre: lustre-OST0000: new disk, initializing [ 3513.651106] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3513.670937] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3515.168377] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3518.136037] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3522.196410] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3522.200235] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3522.219939] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3526.532655] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff in log lustre-MDT0000 [ 3531.771822] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff in log lustre-MDT0000 [ 3537.647074] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb in log lustre-client [ 3577.300566] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3577.300599] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3577.301665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3577.318749] LustreError: Skipped 1 previous similar message [ 3582.309045] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3582.312582] Lustre: Skipped 1 previous similar message [ 3583.002688] Lustre: server umount lustre-OST0000 complete [ 3585.924543] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3585.930583] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3585.938718] Lustre: Skipped 1 previous similar message [ 3585.942163] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3585.946023] Lustre: Skipped 1 previous similar message [ 3587.316654] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3590.428162] Lustre: server umount lustre-MDT0000 complete [ 3591.732843] LustreError: 29679:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744143998 with bad export cookie 14735523118093830930 [ 3591.735900] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3591.746553] LustreError: 29679:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3591.885525] Lustre: server umount lustre-MDT0001 complete [ 3594.915906] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3595.992534] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3597.809470] Lustre: server umount MGS complete [ 3601.553242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3601.719235] LustreError: 4337:0:(qmt_pool.c:1513:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3601.734859] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3602.851408] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3605.812861] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3605.853446] Lustre: 5199:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3605.956690] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3606.990803] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3608.373792] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3609.584574] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3612.121246] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3612.126858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3612.215765] Lustre: 6313:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3613.962213] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3615.764823] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3650.324798] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3650.329422] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3650.337322] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3650.341247] Lustre: Skipped 2 previous similar messages [ 3655.897334] Lustre: server umount lustre-OST0000 complete [ 3661.028743] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3661.037274] LustreError: Skipped 1 previous similar message [ 3661.041585] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3661.054089] Lustre: Skipped 1 previous similar message [ 3661.059502] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3661.063327] Lustre: Skipped 2 previous similar messages [ 3663.387255] Lustre: server umount lustre-MDT0000 complete [ 3664.798274] LustreError: 7319:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744144071 with bad export cookie 14735523118093832757 [ 3664.799839] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3664.805854] LustreError: 7319:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3664.936886] Lustre: server umount lustre-MDT0001 complete [ 3668.776958] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 3669.616091] Key type lgssc unregistered [ 3669.731161] LNet: 8741:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3670.739029] LNet: Removed LNI 192.168.204.136@tcp [ 3677.341175] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 3682.468412] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3682.666686] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3682.671249] alg: No test for adler32 (adler32-zlib) [ 3683.497835] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 3683.559989] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 3683.561377] LNet: Accept secure, port 988 [ 3685.083740] Key type lgssc registered [ 3685.390959] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3688.588501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3690.761012] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3690.769910] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3693.234282] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3695.798550] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3698.232275] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3698.258699] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3698.266854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3699.343810] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 3699.352064] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3699.399336] Lustre: lustre-MDT0000: new disk, initializing [ 3699.429901] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3699.441256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3700.599719] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3704.995734] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3705.001394] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3705.029152] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3705.033871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3705.131594] Lustre: 13365:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3705.148049] Lustre: lustre-OST0000: new disk, initializing [ 3705.151082] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3705.154144] Lustre: Skipped 1 previous similar message [ 3705.183263] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3706.977319] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3707.331548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3707.337373] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3707.351342] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3917.652423] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3917.659825] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3918.660446] Lustre: server umount lustre-MDT0000 complete [ 3920.022934] LustreError: 12417:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744144327 with bad export cookie 5455757909445378130 [ 3920.024719] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3920.035058] LustreError: 12417:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3926.114786] Lustre: server umount lustre-OST0000 complete [ 3934.952980] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_hostid [ 3937.792838] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 3940.996854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3943.457215] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3945.721219] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3945.727270] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3948.084207] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 3948.090781] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3950.629696] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3950.656796] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3950.773119] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000 [ 3950.781257] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3950.832713] Lustre: lustre-MDT0000: new disk, initializing [ 3950.860882] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3950.871347] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3952.024925] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3956.437601] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3956.461472] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3956.502073] Lustre: Modifying parameter lustre-MDT0001.mdt.identity_upcall in log lustre-MDT0001 [ 3956.511984] Lustre: 19665:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3956.532870] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3956.536395] Lustre: Skipped 1 previous similar message [ 3956.589701] Lustre: lustre-MDT0001: new disk, initializing [ 3956.623750] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3956.640838] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3956.647106] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3957.845303] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3961.117982] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3962.345708] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3964.767153] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3964.769717] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3964.787172] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 3964.789669] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3964.854178] Lustre: 20863:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 3964.863080] Lustre: lustre-OST0000: new disk, initializing [ 3964.864636] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3964.880017] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3966.553687] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 3969.729866] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3970.921232] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3974.511328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3974.517917] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3974.540151] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3975.098848] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 3976.294535] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3976.378103] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3979.540839] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3979.540973] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3979.542002] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3979.554955] LustreError: Skipped 1 previous similar message [ 3983.190454] Lustre: server umount lustre-OST0000 complete [ 3986.660708] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3986.663051] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3986.666534] Lustre: Skipped 1 previous similar message [ 3986.668179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3986.671028] Lustre: Skipped 1 previous similar message [ 3989.556627] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3989.556744] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3989.567504] Lustre: Skipped 1 previous similar message [ 3990.670611] Lustre: server umount lustre-MDT0000 complete [ 3992.059037] LustreError: 20888:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744144399 with bad export cookie 5455757909448318907 [ 3992.061436] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3992.072332] LustreError: 20888:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3992.217973] Lustre: server umount lustre-MDT0001 complete [ 3993.483664] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 3994.087099] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 16:33:21 (1744144401) [ 3996.527666] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3996.746255] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3997.932560] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4001.096329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4001.146959] Lustre: 24449:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 4002.430147] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4004.283521] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4005.475357] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4007.987237] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 4007.993026] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4008.096795] Lustre: 25564:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 4009.985061] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4011.863263] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4014.468285] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 4014.475339] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4014.498538] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 4014.504577] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4014.566052] Lustre: lustre-OST0001: new disk, initializing [ 4014.569110] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4014.601228] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4014.605513] Lustre: Skipped 2 previous similar messages [ 4016.424249] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4018.027697] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4018.032884] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4018.048910] LustreError: 27217:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4018.054955] LustreError: 27217:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4019.053325] LustreError: 27221:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4020.061530] LustreError: 27221:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4020.067513] LustreError: 27221:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 4020.144325] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4021.253600] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 4022.069615] LustreError: 27217:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4022.078031] LustreError: 27217:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 4022.496442] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800a92eb000.ost_server_uuid 50 [ 4023.042585] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8800a92eb000.ost_server_uuid in FULL state after 0 sec [ 4024.264781] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800a92eb000.ost_server_uuid 50 [ 4024.762011] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8800a92eb000.ost_server_uuid in FULL state after 0 sec [ 4026.090310] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4038.960423] Lustre: lustre-OST0001: Not available for connect from 192.168.204.36@tcp (stopping) [ 4041.077703] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4041.085454] Lustre: Skipped 1 previous similar message [ 4042.373786] Lustre: server umount lustre-OST0001 complete [ 4046.084832] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4046.086144] LustreError: 26144:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4046.086147] LustreError: 26144:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 2 previous similar messages [ 4050.094574] Lustre: server umount lustre-OST0000 complete [ 4056.100628] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4056.101188] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4056.101190] Lustre: Skipped 4 previous similar messages [ 4056.113512] Lustre: Skipped 3 previous similar messages [ 4057.580913] Lustre: server umount lustre-MDT0000 complete [ 4058.972912] LustreError: 23571:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744144466 with bad export cookie 5455757909448319775 [ 4058.975166] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4058.985495] LustreError: 23571:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4059.127377] Lustre: server umount lustre-MDT0001 complete [ 4063.014075] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 4063.841064] Key type lgssc unregistered [ 4063.947140] LNet: 29620:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4064.958091] LNet: Removed LNI 192.168.204.136@tcp [ 4073.652105] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 4073.745711] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4073.748895] alg: No test for adler32 (adler32-zlib) [ 4074.577521] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 4074.653695] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 4074.656967] LNet: Accept secure, port 988 [ 4076.204683] Key type lgssc registered [ 4076.537420] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4079.294167] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4079.304110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4080.504512] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4081.689871] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4083.572920] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4086.074823] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4086.125870] Lustre: 32135:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 4086.248914] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4087.429817] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4090.921802] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 4090.927991] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4091.020911] Lustre: 593:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 4091.084084] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4092.887038] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4094.770549] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4096.082031] LustreError: 601:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4096.091968] LustreError: 601:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 2 previous similar messages [ 4097.103058] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 4097.109817] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4097.150222] Lustre: 1744:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 4097.217945] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4098.983699] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4100.876401] Lustre: DEBUG MARKER: oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 4105.233551] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4105.240323] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4108.289750] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4108.372928] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4190.373828] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4190.374161] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4190.385382] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4190.389294] Lustre: Skipped 1 previous similar message [ 4193.989758] Lustre: server umount lustre-OST0001 complete [ 4195.380836] LustreError: 601:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4195.389525] LustreError: 601:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 1 previous similar message [ 4196.244582] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4196.249358] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4196.256619] Lustre: Skipped 1 previous similar message [ 4196.259975] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4200.389143] LustreError: 2418:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4200.389245] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4200.389365] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4200.389605] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4200.413548] LustreError: 2418:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 1 previous similar message [ 4201.693715] Lustre: server umount lustre-OST0000 complete [ 4205.396667] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4205.397448] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4205.397451] Lustre: Skipped 1 previous similar message [ 4205.409024] Lustre: Skipped 1 previous similar message [ 4209.205433] Lustre: server umount lustre-MDT0000 complete [ 4210.404374] LustreError: 32144:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4210.413176] LustreError: 32144:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 1 previous similar message [ 4210.637992] LustreError: 2816:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744144617 with bad export cookie 1206685070319757296 [ 4210.640851] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4210.651267] LustreError: 2816:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4210.817445] Lustre: server umount lustre-MDT0001 complete [ 4214.746170] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing unload_modules_local [ 4215.553997] Key type lgssc unregistered [ 4215.655566] LNet: 4266:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4216.663208] LNet: Removed LNI 192.168.204.136@tcp [ 4234.522089] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 4234.686626] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4234.712027] alg: No test for adler32 (adler32-zlib) [ 4235.535057] Lustre: Lustre: Build Version: 2.16.53_41_g5654bc0 [ 4235.606900] LNet: Added LNI 192.168.204.136@tcp [8/256/0/180] [ 4235.610302] LNet: Accept secure, port 988 [ 4237.155637] Key type lgssc registered [ 4237.494400] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4240.737383] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4240.748143] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4241.931461] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4243.104398] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4246.249745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4246.295222] Lustre: 7808:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 4246.417251] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4247.596779] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4248.670185] Lustre: Modifying parameter general.debug_raw_pointers in log params [ 4251.116667] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 4251.123234] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4251.220145] Lustre: 8734:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 4251.276785] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4252.276972] LustreError: 8741:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4252.280380] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 4252.293525] LustreError: 8741:0:(ldlm_lib.c:1095:target_handle_connect()) Skipped 2 previous similar messages [ 4253.067135] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4256.184688] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 4256.190929] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4256.224219] Lustre: 9727:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 4256.289610] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4258.088845] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4260.535027] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 4262.262847] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4267.006561] Lustre: Setting parameter general.lod.*.mdt_hash in log params [ 4276.324733] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4276.326018] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4276.336677] Lustre: Skipped 3 previous similar messages [ 4281.332411] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4281.335721] Lustre: Skipped 6 previous similar messages [ 4281.722859] Lustre: server umount lustre-MDT0000 complete [ 4283.077971] LustreError: 7825:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744144690 with bad export cookie 4411844796160820103 [ 4283.080374] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4283.091680] LustreError: 7825:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4283.247742] Lustre: server umount lustre-MDT0001 complete [ 4294.373185] Lustre: server umount lustre-OST0000 complete [ 4305.848575] Lustre: server umount lustre-OST0001 complete [ 4311.018816] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 4314.103380] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4314.275325] LustreError: 13407:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4314.293650] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4315.458698] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4318.040619] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4318.067010] Lustre: 14270:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 4318.847726] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4319.531441] Lustre: Modifying parameter general.debug_raw_pointers in log params [ 4321.911889] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5 [ 4321.917524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4323.873464] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4324.064060] LustreError: 15202:0:(ldlm_lib.c:1095:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4324.067615] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 4326.954986] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5 [ 4326.961634] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4327.000626] Lustre: 16188:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.136@tcp: no remote llog for lustre-sptlrpc, check MGS config [ 4327.006630] Lustre: 16188:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 1 previous similar message [ 4327.080387] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4327.084341] Lustre: Skipped 2 previous similar messages [ 4328.080569] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 4328.865512] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing set_default_debug -1 all [ 4332.147860] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4333.395287] Lustre: Modifying parameter general.lod.*.mdt_hash in log params [ 4343.109076] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4343.110540] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4343.121571] Lustre: Skipped 3 previous similar messages [ 4345.417403] Lustre: server umount lustre-MDT0000 complete [ 4346.787939] LustreError: 17184:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744144753 with bad export cookie 4411844796160821790 [ 4346.790963] LustreError: MGC192.168.204.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4346.800878] LustreError: 17184:0:(ldlm_lockd.c:2591:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4346.943870] Lustre: server umount lustre-MDT0001 complete [ 4358.423582] Lustre: server umount lustre-OST0000 complete [ 4369.920935] Lustre: server umount lustre-OST0001 complete [ 4371.256349] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 4371.828403] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 4372.430167] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 16:39:39 (1744144779) [ 4379.581815] Lustre: DEBUG MARKER: oleg436-server.virtnet: executing load_modules_local [ 4382.245804] SGI XFS with ACLs, security attributes, no debug enabled [ 4382.251970] XFS (loop0): Mounting V5 Filesystem [ 4382.264636] XFS (loop0): Ending clean mount [ 4383.282373] Lustre: DEBUG MARKER: conf-sanity test_116: @@@@@@ FAIL: failed to mkfs for /tmp/d116.conf-sanity/f116.conf-sanity-mdt0