== conf-sanity test 24a: Multiple MDTs on a single node == 10:23:37 (1761315817) pdsh@oleg231-client: oleg231-server: ssh exited with exit code 1 Permanent disk data: Target: 969362ae:MDT0000 Index: 0 Lustre FS: 969362ae Mount type: zfs Flags: 0x61 (MDT first_time update ) Persistent mount opts: Parameters: sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity mgsnode=192.168.202.131@tcp:192.168.202.131@tcp mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-mdt1_2 /tmp/lustre-mdt1_2 mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-mdt1_2/mdt1_2 xattr=sa dnodesize=auto compression=off Writing lustre-mdt1_2/mdt1_2 properties lustre:sys.timeout=20 lustre:mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity lustre:mgsnode=192.168.202.131@tcp:192.168.202.131@tcp lustre:version=1 lustre:flags=97 lustre:index=0 lustre:fsname=969362ae lustre:svname=969362ae:MDT0000 Permanent disk data: Target: 969362ae:OST0000 Index: 0 Lustre FS: 969362ae Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.202.131@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1_2 /tmp/lustre-ost1_2 mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1_2/ost1_2 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1_2/ost1_2 properties lustre:mgsnode=192.168.202.131@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=969362ae lustre:svname=969362ae:OST0000 start mds service on oleg231-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg231-server' oleg231-server: oleg231-server.virtnet: executing load_modules_local oleg231-server: Loading modules from /home/green/git/lustre-release/lustre oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg231-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg231-server: detected 4 online CPUs by sysfs oleg231-server: Force libcfs to create 2 CPU partitions oleg231-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg231-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg231-server: oleg231-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg231-client: oleg231-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg231-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg231-server: oleg231-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg231-client: oleg231-server: ssh exited with exit code 1 Started lustre-OST0000 oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg231-client.virtnet: -o user_xattr,flock oleg231-server@tcp:/lustre /mnt/lustre Starting fs2mds: -o localrecov lustre-mdt1_2/mdt1_2 /mnt/lustre-fs2mds oleg231-server: oleg231-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg231-client: oleg231-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1_2/mdt1_2 Started 969362ae-MDT0000 Starting fs2ost: -o localrecov lustre-ost1_2/ost1_2 /mnt/lustre-fs2ost oleg231-server: oleg231-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg231-client: oleg231-server: ssh exited with exit code 1 Commit the device label on lustre-ost1_2/ost1_2 Started 969362ae-OST0000 969362ae-MDT0000: Deregistered changelog user #1 setup single mount lustre success cp: cannot create regular file '/mnt/lustre2/f24a.conf-sanity': No space left on device conf-sanity test_24a: @@@@@@ FAIL: cp /etc/passwd /mnt/lustre2/f24a.conf-sanity failed Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6482:error() = /home/green/git/lustre-release/lustre/tests/conf-sanity.sh:1018:test_24a() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6840:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6887:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6713:run_test() = /home/green/git/lustre-release/lustre/tests/conf-sanity.sh:1035:main() Dumping lctl log to /tmp/testlogs//conf-sanity.test_24a.*.1761315958.log umount /mnt/lustre2 ... stopping fs2mds ... rsync: chown "/tmp/testlogs/.conf-sanity.test_24a.debug_log.oleg231-server.1761315958.log.oSa8KA" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.conf-sanity.test_24a.dmesg.oleg231-server.1761315958.log.MHwtkg" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] Stopping /mnt/lustre-fs2mds (opts:-f) on oleg231-server stopping fs2ost ... Stopping /mnt/lustre-fs2ost (opts:-f) on oleg231-server