== sanity-quota test 7c: Quota reintegration (restart mds during reintegration) ========================================================== 10:16:44 (1773670604) Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 Waiting 90s for 'none' fail_val=0 fail_loc=0xa03 Waiting 90s for 'ugp' osd-ldiskfs.lustre-OST0000.quota_slave.force_reint=1 osd-ldiskfs.lustre-OST0001.quota_slave.force_reint=1 Stop mds... Stopping /mnt/lustre-mds1 (opts:) on oleg150-server fail_val=0 fail_loc=0 Start mds... Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg150-server: oleg150-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg150-client: oleg150-server: ssh exited with exit code 1 Started lustre-MDT0000 affected facets: ost1 oleg150-server: oleg150-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg150-server: *.lustre-OST0000.recovery_status status: COMPLETE Waiting 200s for 'glb[1],slv[1],reint[0]' Waiting 190s for 'glb[1],slv[1],reint[0]' Waiting 180s for 'glb[1],slv[1],reint[0]' Waiting 170s for 'glb[1],slv[1],reint[0]' Waiting 160s for 'glb[1],slv[1],reint[0]' Waiting 150s for 'glb[1],slv[1],reint[0]' Waiting 140s for 'glb[1],slv[1],reint[0]' Waiting 130s for 'glb[1],slv[1],reint[0]' Waiting 110s for 'glb[1],slv[1],reint[0]' Waiting 100s for 'glb[1],slv[1],reint[0]' Updated after 107s: want 'glb[1],slv[1],reint[0]' got 'glb[1],slv[1],reint[0]' affected facets: ost2 oleg150-server: oleg150-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg150-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg150-server: oleg150-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg150-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg150-server: oleg150-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg150-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg150-server: oleg150-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg150-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg150-server: oleg150-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg150-server: *.lustre-OST0001.recovery_status status: INACTIVE running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/zero] [bs=1M] [of=/mnt/lustre/d7c.sanity-quota/f7c.sanity-quota] [count=6] [oflag=sync] dd: error writing '/mnt/lustre/d7c.sanity-quota/f7c.sanity-quota': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.4476 s, 9.4 MB/s Delete files... Wait for unlink objects finished... Waiting for MDT destroys to complete